Using software architecture for code testing - Software ...
Upcoming SlideShare
Loading in...5

Using software architecture for code testing - Software ...






Total Views
Views on SlideShare
Embed Views



1 Embed 1 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Using software architecture for code testing - Software ... Using software architecture for code testing - Software ... Document Transcript

  • 160 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 30, NO. 3, MARCH 2004 Using Software Architecture for Code Testing Henry Muccini, Antonia Bertolino, and Paola Inverardi, Member, IEEE Abstract—Our research deals with the use of Software Architecture (SA) as a reference model for testing the conformance of an implemented system with respect to its architectural specification. We exploit the specification of SA dynamics to identify useful schemes of interactions between system components and to select test classes corresponding to relevant architectural behaviors. The SA dynamics is modeled by Labeled Transition Systems (LTSs). The approach consists of deriving suitable LTS abstractions called ALTSs. ALTSs offer specific views of SA dynamics by concentrating on relevant features and abstracting away from uninteresting ones. Intuitively, deriving an adequate set of test classes entails deriving a set of paths that appropriately cover the ALTS. Next, a relation between these abstract SA tests and more concrete, executable tests needs to be established so that the architectural tests derived can be refined into code-level tests. In the paper, we use the TRMCS case study to illustrate our hands-on experience. We discuss the insights gained and highlight some issues, problems, and solutions of general interest in architecture-based testing. Index Terms—Software engineering, software architectures, testing and debugging, testing strategies, tracing. æ 1 INTRODUCTION I N recent years, the study of software architecture (SA) has emerged as an autonomous discipline requiring its own concepts, formalisms, methods, and tools [30], [3], [24]. SA been proposed to select tests from semiformal [46], algebraic [5], and model-based [19] specifications. Recently, various approaches have been proposed on represents a very promising approach since it handles the testing driven by the architectural specification [28]. Among design and analysis of complex distributed systems and these, we have been addressing the problem of identifying tackles the problem of scaling up in software engineering. suitable functional test classes for the testing in-the-large of Through suitable abstractions, it provides the means to complex real-world systems [8], [9]. This paper builds on make large applications manageable. SAs support the our previous work [8], [9], presenting the fully developed formal modeling of a system, allowing for both a approach in a coherent, comprehensive way. topological (static) description and a behavioral (dynamic) Our goal is to provide a test manager with a systematic one. SA models the system in terms of components and method to extract suitable test classes for the higher levels connectors, where components represent abstract computa- of testing and to refine them into concrete tests at the code tional subsystems and connectors formalize the interactions level. Our approach is based on the specification of among components. Both industry and academia have SA dynamics, which is used to identify useful schemes of actively used SA to improve the dependability of complex interactions between system components and to select test systems [3], [24] and SA-based processes have been classes corresponding to relevant architectural behaviors. proposed to model and analyze real systems [3], [30]. Our SA dynamics is modeled by Labeled Transition A crucial part of the development process is testing: Systems (LTSs). The approach consists of deriving suitable Software testing consists of the dynamic verification of a LTS abstractions, called Abstract LTSs or ALTSs. ALTSs program’s behavior performed by observing the execution offer specific views of the SA dynamics by concentrating of the program on a selected set of test cases [6]. In on relevant features and abstracting away from uninterest- particular, specification-based testing checks that the Im- ing ones. Intuitively, deriving an adequate set of test plementation Under Test (IUT) fulfills the specifications, classes entails deriving a set of paths that appropriately used to “capture” all and only the important properties cover the ALTS. Then, the architectural tests must be against which the IUT has to be tested. refined into code-level tests in order to be executed. To this The importance of the use of formal methods in software specification and design does not need to be stressed. end, here we follow a stepwise, manual methodology to Several authors have highlighted the advantages of formal deal with the lack of a formal relation between the SA methods in testing as well, and several techniques have description and the code. Note that our approach differs from, for example, applying a coverage criterion directly over the complete . H. Muccini and P. Inverardi are with the Dipartimento di Informatica, LTS. Indeed, the ALTS is a tool to understand what must be ´ Universita dell’Aquila, Via Vetoio 1, 67100 L’Aquila, Italy. E-mail: {muccini, inverard} tested. In other words, it represents a model of the system, . A. Bertolino is with the Istituto di Scienza e Tecnologie dell’Informazione whereas a coverage criterion is just a method to efficiently “A. Faedo” (ISTI-CNR), Area della Ricerca CNR di Pisa, 56100 Pisa, produce a set of test scenarios from the specified model. Italy. E-mail: The contribution of this paper is twofold: 1) it develops a Manuscript received 9 June 2003; revised 6 Nov. 2003; accepted 9 Jan. 2004. methodology for the extraction of test specifications Recommended for acceptance by J. Knight. For information on obtaining reprints of this article, please send e-mail to: relevant at an architectural level and 2) it tests out the, and reference IEEECS Log Number TSE-0068-0603. viability of the approach in a real-world context. The 0098-5589/04/$20.00 ß 2004 IEEE Published by the IEEE Computer Society
  • MUCCINI ET AL.: USING SOFTWARE ARCHITECTURE FOR CODE TESTING 161 empirical investigation with a case study was instrumental Steps 1 to 3 present a rigorous method to extract for the latter goal. architectural tests from an SA specification. Step 4 deals The remainder of this paper is organized as follows: with the execution of these tests at the code level. As far as Section 2 presents an overview on the approach. Section 3 this paper is concerned, we could not rely on a strictly outlines the case study to which we apply the approach. formal refinement process from SA to code. So, the last step Section 4 describes the approach in detail, and Section 5 is dealt with using a less formal approach than the first presents its application to the case study. Sections 4 and 5 three. Indeed, applying the approach to the case study is not may be read concurrently: They are organized to present an a posteriori validation, but is instrumental in under- the theory and practice, respectively, of each step of our standing the issues that concretely arise in the execution of approach. Section 6 discusses some problems and insights SA-based tests at code-level, as well as in working out a gained, and Section 7 considers related work. Section 8 systematic procedure to address them. summarizes the paper and presents future work plans. 3 THE TRMCS CASE STUDY 2 AN OVERVIEW OF OUR APPROACH TO SA The Teleservice and Remote Medical Care System (TRMCS) TESTING [32] is a system that provides monitoring and assistance to Several Architectural Description Languages (ADLs) rely disabled or elderly people. A typical service is to send on Labeled Transition Systems to model the SA dynamics relevant information to a local phone center so that the [31], [38], [36], [24]. In our approach, we assume the family, along with those concerned with medical or existence of an SA description, in some ADL, and that an technical assistance, can be timely notified of critical LTS model can be derived from such a description whose circumstances. node and arc labels represent, respectively, states and From the informal requirements of TMRCS, we derive an transitions relevant in the context of SA dynamics. Note that SA model by defining the system topology (static view of the LTS model expresses aspects such as concurrency, components and connectors) and the system dynamics (the nondeterminism, asynchrony, and so on, relative to interactions). architectural nonsequential behaviors. We recall the defini- The resulting TRMCS software architecture is shown in tion of an LTS. Fig. 1. In it, boxes represent components (i.e., processing elements), arrows identify connectors (i.e., connecting Definition 1. A Labeled Transition System (LTS) is a elements, in this case channels), and arrow labels refer to quintuple ðS; L; S0 ; SF ; T Þ, where S is the set of states, L the data elements exchanged through the channels. In is the set of distinguished labels (actions) denoting the LTS particular, we identify four component types: alphabet, S0 2 S is the initial state, SF  S is the set of l final states, and T ¼ fÀ  S Â S j l 2 Lg is the transi- ! . User sends either an “alarm” or a “check” message tion relation labeled with elements of L. to the Router process. After sending an alarm, he/ she waits for an acknowledgment from the Router. In the following, we use the terms LTS “labels” and In our analysis, we consider the case in which two “actions” interchangeably. Users can concurrently send alarms and checks. l1 l2 l3 A path p on an LTS, where p ¼ S0 À S1 À S2 À . . . ! ! ! . Router waits for signals (check or alarm) from the ln À Sn , is a complete path if S0 is the initial state and Sn is ! User. It forwards the alarm messages to the Server a final state. Hereafter, for the sake of brevity, an LTS and monitors the state of the User via check path will also be denoted by its sequence of labels (e.g., messages. p ¼ l1 :l2 : . . . :ln ). . Server dispatches the help requests. Our approach to SA-based testing consists of four . Timer sends a clock signal at each time unit. logical steps. As the starting point we assume that the The Java implementation of the TRMCS was devel- software architect, by looking at the SA dynamics from oped independently on the basis of the same informal different viewpoints, defines various obs-functions over requirements. the SA model, each one highlighting a specific perspec- This case study serves three main purposes: to empiri- tive of interest for a test session (Step 1). Applying each cally validate the feasibility of the approach as a whole, to obs-function to the LTS, an Abstract LTS (ALTS) can be specifically work out the procedure relative to Step 4, as generated which is a reduced LTS showing only interest- stated above, and, finally, to verify to what extent the ing behaviors with respect to the selected view (Step 2). approach is able to identify conformance errors between the On this graph, which is much more manageable than the SA reference model and the implementation. original LTS, the software architect chooses a set of important patterns of behaviors (paths over the ALTS) to be tested (Step 3). Finally, these high-level tests are passed 4 THE APPROACH to the software tester, who actually has to run the tests In this section, we describe the four steps of our approach to and observe whether the current implementation “con- SA-based testing. forms” to its architectural model (Step 4). We say that the implementation does not conform to the specification if some 4.1 Step 1: Obs-Functions Definition interactions described at the architectural level would not be From the SA specification of the system under analysis, allowed in the implementation. we derive a Labeled Transition System that models the
  • 162 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 30, NO. 3, MARCH 2004 Fig. 1. TRMCS components and connectors. SA dynamics. In principle, this graph could directly be used We can also extend the obs-function definition to LTS as the reference model for deriving the test scenarios. The paths so that, if p ¼ l1 :l2 : . . . :ln , problem is that the LTS provides a global, monolithic obsðpÞ ¼ obsðl1 :l2 : . . . :ln Þ ¼ obsðl1 Þ:obsðl2 Þ: . . . :obsðln Þ: description; it is a vast amount of information flattened into a graph. Extracting from this global model the observations 4.2 Step 2: Abstract LTS Derivation of system behavior that are relevant for validation is a We use the obs-function as a means to derive a smaller difficult task. This is a problem that always arises in formal automaton from the LTS which still expresses all high-level testing: With the exception of very small routines, we need behaviors we want to test according to the selected ways for exploring the LTS and deriving representative SA testing criterion, but hides away any other irrelevant behaviors that constitute the test suite. behavior. The automaton is called an Abstract LTS (ALTS). Our aim is to provide software architects with a key to The ALTS is obtained from the LTS via two transforma- decipher the LTS. A common practice in the analysis of tions: 1) by relabeling each transition in the LTS according to complex systems is to derive from the global SA model a set the obs-function and 2) by minimizing the resulting auto- of simplified models that provide different system views. maton with respect to a selected equivalence relation. We Accordingly, the basic idea of our approach is to allow for analyzed trace and bisimulation-based equivalences, both the formal derivation from the LTS of reference models for familiar from the theory of concurrency [42]. If one wants to testing, each representing a relevant pattern of behavior; reduce as much as possible the number of  transitions and with some abuse of terminology, we refer to each of the corresponding nodes, then a trace equivalence can be selected patterns of behavior as an SA testing criterion. considered. In fact, this equivalence abstracts from Intuitively, an SA testing criterion abstracts away interac- -labeled transitions and concentrates on any computational tions that are uninteresting with regard to a specific test paths of transitions that are different from . A bisimulation- concern. Referring to the LTS definition, an SA testing based equivalence is more suitable when one wants to criterion naturally partitions the LTS actions L into two observe how the system evolves step-by-step, even along groups: relevant interactions R (i.e., those we want to -moves (preserving the branching structure). The relabeled automaton is called the ObsLTS and the observe by testing) and nonrelevant interactions N R (i.e., reduced one is the ALTS. those we are not interested in at this time) so that L ¼ Fig. 2 gives an example of the ALTS construction: The R [ N R and R N R ¼ ;. original LTS is analyzed, identifying the observation of We therefore associate with an SA testing criterion an interest (Fig. 2a); the abstraction is applied over this LTS obs-function that maps the relevant LTS labels to a specified with respect to the selected obs-function (Fig. 2b); the trace interpretation domain D, whereas any other (nonrelevant) equivalence minimization function is applied. The resulting one is mapped to a distinct element . The obs-function may ALTS is shown in Fig. 2c. Fig. 2d, in contrast, presents a also be considered as a hiding operator that makes a set of (bisimulation-based) branch minimization. As can be seen, actions invisible to its environment and may relabel the the branch minimization is more informative since it gives others in an interpretation domain D. The relabeling may more data on the original LTS structure. help emphasize the semantic meaning of observable Taking into consideration that 1) the aim of ALTS is to actions. More precisely: provide a more compact and analyzable graph, and 2) the obs : LÀ 0 ; so that obsðr 2 RÞ ¼ d 2 D; !L ALTS automaton is built to highlight only interesting behaviors, the trace equivalence is more suitable for our obsðn 2 N RÞ ¼ ; and L0 ¼ D [ : purposes. Moreover, full information about the LTS may be
  • MUCCINI ET AL.: USING SOFTWARE ARCHITECTURE FOR CODE TESTING 163 Fig. 2. (a) LTS. (b) ObsLTS. (c) Trace-based ALTS. (d) Bisimulation-based ALTS. retrieved (if necessary) by using the algorithm explained in comparing various criteria applied over the same SA, is [43], Appendix A. outside the scope of this paper. The ALTS generation process1 can be proven to be correct When considering what to take as the specification of an and complete, that is, each ALTS path comes from an LTS path “architectural test,” we are left with two options. The first (ALTS does not introduce new paths) and each LTS path can option, which is the one we take in this paper, is to consider be mapped onto an ALTS path (ALTS does not loose an ALTS complete path as the test specification. In this case, information), as discussed in [43], Appendix B. the test is specified at a more abstract level and the tester instinctively focuses the testing on a restricted set of 4.3 Step 3: Test Selection interactions (those in the ALTS alphabet). A second option Once the ALTS associated with an SA testing criterion via could be to identify those LTS paths of which the selected an obs-function has been derived, the task of deriving an ALTS path is an abstraction (this process is illustrated in adequate set of tests according to the selected test criterion [43], Appendix A). Because LTS paths are more detailed is converted into the task of deriving a set of complete paths than ALTS paths, in this case, the tester would have more appropriately covering the ALTS. information about how to perform the tests, but also stricter Given the relatively small dimensions of an ALTS, a requirements, that is, the tester doesn’t have as much tester might consider applying extensive coverage criteria freedom in choosing the code-level tests. In practice, it on it. However, ALTS paths clearly specify architectural might actually be more difficult to test the conformance of behaviors at a higher abstraction level than the original LTS source code to the test specification. because they are based on an obs-function applied over the In either case (ALTS or LTS path), an architectural test is LTS. Thus, one ALTS complete path can correspond to essentially a sequence of system actions that are meaningful at the SA level. many possible LTS paths (i.e., architectural tests). Therefore, less thorough coverage criteria seem more practical. 4.4 Step 4: Code-Level Testing Here, we apply McCabe’s [40] and Watson and McCa- In this section, we discuss how a tester can use the be’s [54] test technique as it is a good compromise between architectural paths, as produced in Section 4.3, to test the arc and path coverage in the case of ALTS coverage; in fact, implementation. In general, when the test cases are derived any base set of paths covers all edges and nodes in the from an analysis of the specifications, two major problems graph (i.e., this coverage subsumes branch and statement remain to be solved: traceability and test execution. coverage testing). Traceability concerns “relating the abstract values of the Clearly, any ALTS coverage criteria, such as McCabe’s or specification to the concrete values of the implementation” other potential alternatives, need to be evaluated empiri- [19]. Here, we are concerned with traceability from the cally, considering the cost and effectiveness of the obtained SA-level of abstraction. Several researchers have recognized test sets (i.e., the number of test sequences and their the importance and difficulty of this problem [55], [48], but respective effectiveness in detecting potential nonconfor- no one has yet found a general solution. mance failures). This evaluation, which would entail If a well-formalized architecture-based development process is in place, SA specifications can be used to 1. The ALTS derivation is completely automated: We customized the formally drive the generation of low-level design and code existing FC2Tools [21], taking advantage of the functions “fc2explicit” and “fc2weak.” A short description of the tools that support our approach can and, thus, the correspondence is maintained throughout be found in [43], Appendix C. the process. For instance, some ADLs (such as C2ADL [12]
  • 164 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 30, NO. 3, MARCH 2004 and xADL [18]) provide development support for im- this context, “acti before actj ” does not mean that all the plementing software architectures in Java and C++ [13]. operations implementing acti must run before all the Explicit mapping rules drive the source-code implementa- operations implementing actj . It means that some opera- tion of architectural components, connectors, and mes- tions that identify the action termination must be executed sages via objects. following a given order, whereas the others may be run in Considering specifically the process followed in the any order. In Section 5, we further develop this idea in the experience described in this paper, due to real-world context of the TRMCS system, and the operations identify- constraints, SA specifications and low-level design have ing the termination of actions are denoted as Synchroniza- been intermixed without any formal mapping, but using tion Methods (SMs). ad hoc monitoring and code-analysis techniques. While Step 4.3. At this point, we run the code and evaluate the this is certainly not the ideal process, it is a realistic and execution traces with respect to the expected ones (i.e., plausible approach, and we discuss some interesting those identified in Step 4.2) to analyze the source code insights gained. The procedure we followed consists of conformance with respect to the architectural behavior. To Step 4.1 and Step 4.2, described later in this section. execute the desired test sequence, our pragmatic approach Test execution entails forcing the Implementation Under is to repeat the launching of a program run under some Test (IUT) to execute the specific sequence of events that has specified input conditions several times until the sequence been selected. This is a difficult task, which becomes harder is observed (or a maximum number of iterations is in distributed and concurrent settings, where nondetermin- reached). ism is introduced. In fact, depending on the system state, a In the literature, this is called a nondeterministic testing same input might excite different sequences of interactions approach. In contrast, the deterministic testing approach (first (among several concurrent processes) and produce different proposed by Carver and Tai [16]) forces a program to results. A detailed discussion of this problem is outside the execute a specified test sequence by instrumenting it with scope of this paper. In brief, one can distinguish between synchronization constructs that deterministically reproduce deterministic and nondeterministic-testing approaches, as the desired sequence. Instrumentation is commonly used in discussed below in Step 4.3. software testing [28], but care must be taken to contain the In the following, we show how Step 4 is divided into consequent execution overhead and avoid the triggering of three substeps, explaining how we tackled both problems. unexpected behaviors. Alternatively, a language-based Step 4.1. In this substep, we map architectural compo- dynamic approach for Java requiring no instrumentation nents and actions (identified by the LTS labels) into their has been recently proposed in [35]. A test driver auto- low-level implementation. As mentioned before, our matically executes the calls as specified in a test sequence, experience was conducted in a realistic context in which controlling the synchronization of the Java threads through we could not make any assumptions on the existence of a clock. A related, yet different problem is execution replay formal relationships between software architecture and for reproducing an observed trace, as required, e.g., during design elements. We thus analyzed the system implemen- debugging to force a series of synchronization events that tation to understand how architectural actions (e.g., high- led to a failure. A tool for the deterministic replay of Java level functionalities) have been implemented in the code multithreaded applications is presented in [17]. by sequences of partially ordered method calls. Note that more than one implementation sequence might corre- 5 APPLYING THE APPROACH spond to one LTS action. In such cases, to test the We now apply the approach introduced in the previous architectural action, all of them should be considered. Of section to the TRMCS case study presented in Section 3. We course, the same implementation action (e.g., a method specify the TRMCS components behavior by using the call) may occur in several implementation sequences, Finite State Process (FSP) [38] language. The behavior of corresponding to different architectural actions. This is not each component is modeled by one or more FSP processes, a problem, however, because we test sequences and not and each process is described by an LTS which is single implementation actions. automatically generated by the Labeled Transition System Step 4.2. This second substep takes into account how an Analyzer (LTSA) tool [34]. ordered sequence of actions (i.e., an architectural test) is By running the LTSA tool on the TRMCS FSP specifica- implemented by a sequence of low-level functions. Since the tion, we obtain a (minimized) LTS composed of 256 states. TRMCS implementation is in Java, we map actions into The LTS labels of interest for our analysis are: sequences of method calls. If each action is implemented, at the low level, by a sequence of methods calls, it would be 1. u[i].sendAlarm_To_Router: An Alarm msg is sent useful to understand how sequences of these actions (i.e., an by Useri to the Router. architectural test) are implemented by the source code. 2. u[i].receiveAck_From_Router: The Router Ac- Two alternatives may be taken into account: 1) Each knowledgment msg is received by Useri . action is implemented by a sequence of operations and they 3. r.[i].sendAlarm_To_Server: The Router forwards run sequentially, or 2) the actions can run concurrently. In the Alarm msg to the Server. the former case, a sequence of architectural actions is 4. sa[0].sendAck_To_Router: The Server sends the Ack implemented by the sequential execution of the respective to the Router. low-level sequence diagrams. In the latter case, the 5. r.sendNoFunc_To_Server: The Router sends the operations may interleave with each other. Note that, in NoFunction msg to the Server.
  • MUCCINI ET AL.: USING SOFTWARE ARCHITECTURE FOR CODE TESTING 165 receiving are relabeled (Alarmi Dispatch, Acki Dispatch) and any other is hidden. An alternative scenario could be that the TRMCS is already functioning and one of the components is being modified. We then want to test whether the modified component still interacts with the rest of the system in conformance with the original description of the SA. In this case, the observation point of the software architect is “all the interactions that involve this component.” If, for instance, the component being modified is specifically the Server, then the corresponding obs-function is the one given in Fig. 3b: FR{a1, a2, no} is used to relabel the messages the Server receives from the Router, and TR{ack1, ack2} is used to relabel the messages sent to the Router. This SA testing criterion is referred to hereafter as “ServerRegression.” 5.2 Applying Step 2 With reference to the AlarmObs criterion, after applying Fig. 3. (a) AlarmObs obs-function. (b) ServerRegression obs-function. reduction and minimization algorithms, we obtain the ALTS depicted in Fig. 4a (the shaded circle represents the The steps presented in the following description are the initial state, which, in this example, also coincides with the same as those presented in the previous section. only final one). This ALTS represents, in a concise, 5.1 Applying Step 1 graphical way, how the Alarm flow is handled: After an Given the informal description of the TRMCS in Section 3, Alarm is issued (e.g., Alarm1Dispatch), the system can due to obvious safety-critical concerns, we may want to test nondeterministically react with one of two possible actions: the flow of an Alarm message, from the moment a User 1) elaborate this Alarm and send back an Acknowledgment sends it to the moment the User receives an acknowl- (Ack1Dispatch), or 2) receive another Alarm message from edgment. In the terminology used in Section 4, the software another User (Alarm2Dispatch). architect may decide that an important SA testing criterion With reference to the ServerRegression criterion, the ALTS in Fig. 4b is obtained. It shows the interactions is “all those behaviors involving the flow of an Alarm message involving the Server component: It can receive Alarm1 through the system.” (“FRa1”) or Alarm2 (“FRa2”) from the Router and, after- From this quite informal specification, a corresponding ward, it can nondeterministically receive other Alarms or obs-function can be formally defined. This is called send back Acknowledgments (“TRacki ”). At any moment, “AlarmObs” and is shown in Fig. 3a. The set of relevant it can receive the NoFunc message from the Router actions R for AlarmObs contains all of the actions (i.e., (“FRno”). elements of the LTS alphabet) that specifically involve sending an Alarm message by a User (label 1 above) or the 5.3 Applying Step 3 User’s subsequent reception of the Router acknowledgment With reference to the ALTS derived for the AlarmObs (label 2). The corresponding actions must be relabeled in D criterion (Fig. 4a), a list of test paths derived according to according to their semantic interpretation. As shown in Watson and McCabe’s coverage criterion [54] follows with Fig. 3a, the four actions involving Alarm sending and Ack the corresponding Test Sequences (TSs): Fig. 4. (a) AlarmObs ALTS. (b) ServerRegression ALTS.
  • 166 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 30, NO. 3, MARCH 2004 Fig. 5. The implementation of the AlarmDispatch. (a) User send alarm scenario. (b) Router receive alarm scenario. . Path1a : A B A, TS1a : Alarm1Dispatch.Ack1Dispatch. Paths 1b , 3b , 7b , and 8b , for example, are meant to verify . Path2a : A B D B A, TS2a : Alarm1Dispatch.Alarm2 that the NoFunc message (FRno in Fig. 4b) can be received Dispatch.Ack2Dispatch.Ack1Dispatch. by the Server at any moment. . Path3a : A B A C A, TS3a : Alarm1Dispatch.Ack1Dis patch.Alarm2Dispatch.Ack2Dispatch. 5.4 Applying Step 4 . Path4a : A B D C A, TS4a : Alarm1Dispatch.Alarm2 We now concentrate on the test sequences generated from Dispatch.Ack1Dispatch.Ack2Dispatch. the AlarmObs ALTS in Section 5.3. These tests involve three . Path5a : A B D C D C A, TS5a : Alarm1Dispatch. different components (User1, User2, and Router) and two Alarm2Dispatch.Ack1Dispatch.Alarm1Dispatch. architectural events (AlarmDispatch and AckDispatch). By Ack1Dispatch.Ack2Dispatch. following the three steps defined in Section 4.4, we have: Let us consider, for example, Paths 2a , 3a , and 4a . These Applying Step 4.1: AlarmDispatch and AckDispatch are three paths are aimed at validating that no Alarm messages the events of interest when considering the selected in a series of two is lost, irrespective of the order in which architectural tests. We need to understand how the source they are processed. Moreover, for conformance testing code implements these architectural actions. purposes, we want to check whether the system imple- The AlarmDispatch operation represents a system Input, mentation enables all the sequences of actions envisioned at and what we expect is that the alarm process is started the architectural level. when the User pushes the Alarm button of the hardware With reference to the ServerRegression observation and device. Following this reasoning, we start by analyzing the the associated ALTS (Fig. 4b), McCabe’s coverage criterion actions associated with pushing the Alarm button, and we yields the following set of test paths with the corresponding find that the Alarm functionality is implemented by two TS lists: different Java processes: The Alarm message is written into a socket and the Router periodically reads the socket . Path1b : A A, TS1b : Frno. looking for new messages. The Ack function is implemen- . Path2b : A B A, TS2b : FRa1.TRack1. ted following a similar scheme: The Router writes into a . Path3b : A B B A, TS3b : FRa1.FRno.TRack1. socket and the User waits on a defined socket port. . Path4b : A B D B A, TS4b : FRa1.FRa2.TRack2.TRack1. The objects and the methods that implement the Alarm . Path5b : A B A C A, TS5b : FRa1.TRack1.FRa2.TRack2. are shown in Fig. 5. Let us very briefly comment on it: . Path6b : A B D C A, TS6b : FRa1.FRa2.TRack1.TRack2. From the User side (Fig. 5a), the xUser object creates the . Path7b : A B D D B A, TS7b : FRa1.FRa2.FRno. User object if the if_1 condition is verified. After some TRack2.TRack1. time (in which internal operations can be performed), if . Path8b : A B D C C A, TS8b : FRa1.FRa2.TRack1.FRno. Users press the Alarm button (if_2), they thereby create a TRack2. ClientConnection and the SendNMessages (SNM) objects. . Path9b : A B D C D B A, TS9b : FRa1.FRa2.TRack1. The User calls the send() method of the SNM object, FRa1.TRack2.TRack1. which tries “n” times to create the socket. If the socket is
  • MUCCINI ET AL.: USING SOFTWARE ARCHITECTURE FOR CODE TESTING 167 created (if_3), the SNM finally writes the Alarm message about the execution of the SMs. To implement this new into the socket. From the Router side (Fig. 5b), the Router feature, we introduce the “SynchronizationMethod” file; graphical interface object (xRouter) creates a new instance when an SM is called by the system, a print operation is of the MasterRouter object, which in turn creates the performed in this file which captures the following ServerConnection object. It tries to create a ServerSocket information: <SM ID, User ID, SM instance, time>. For “ss” and, then, a process is activated that continuously example, <AlarmSM, User 1, Alarm 3, 974910169250> checks the ServerSocket port. In this way, when a new records that the synchronization method AlarmSM (in message is written in the socket (if_b) the socket is read Fig. 5) has been called (at the system time 974910169250) and the Alarm is received. in response to the third instance of an Alarm message A similar analysis has been conducted for the Ack issued by User1. function. By running P 0 several times with the User1 and User2 Applying Step 4.2. Now, we will analyze how an Alarms as inputs in different orders, we obtain a Synchro- architectural test (i.e., a sequence of actions) is imple- nizationMethod file report. The information collected by the mented by the code. Let us consider as an example TS3a , report is a sequence of pairs of lines such as the following Alarm1Dispatch. Ack1Dispatch. Alarm2Dispatch. Ack2- couple: Dispatch. The actions of interest in this test sequence are Alarmi Dispatch and Acki Dispatch (for i ¼ 1; 2), and their . . . ; < AlarmSM; Useri; Alarmj; ti >; order is such that Alarm1Dispatch happens before < AckSM; Useri; Alarmj; ti þ  >; . . . Ack1Dispatch, which happens before Alarm2Dispatch, which happens before Ack2Dispatch. In other words, each Alarm operation is always im- At the architectural level, Alarm1Dispatch happens mediately followed by the relative Acknowledgment, as before Ack1Dispatch if the second action is run after the described by behavior TS3a in Section 5.3. This implies that first one has terminated. At the code level, the ordering we never observe behaviors TS2a and TS4a . Triggered by relation “happens before” can be interpreted in two ways: this observation, code analysis pictures a conformance fault with respect to the architectural expected behaviors. 1. If the code-level sequences implementing the Alarmi Dispatch (Fig. 5) and the Acki Dispatch actions are run sequentially, Alarm1Dispatch happens before 6 SOME CONSIDERATIONS Ack1Dispatch if the Alarm scenarios in Fig. 5 are run Considering the four steps of our approach, it is possible to to completion before the Ack scenarios start. make an empirical evaluation of the difficulty of applying it 2. If the low-level scenarios can be concurrently run, and its generality, which can then lead to further lines of their method calls may interleave. In particular, if research. the implementations for Alarmi and Acki run concurrently, the methods in their scenarios could 6.1 Difficulty of Application be called in a different order. In this context, then, Step 1. Obs-functions definition: The definition of an what does “Alarmi happens before Acki ” mean? To obs-function of interest is an empirical task, based on a answer this question, we identify those methods software architect’s experience. This task could be made (Synchronization Methods or SMs) that actually easier through a classification of observations and by perform an architectural action. Therefore, Alarm1 assigning semantics to the messages exchanged in the SA before Ack1 means that the Alarm1 SM happens model. Methods similar to SAAM [33] or SCENT [49], in before the Ack1 SM, whereas other methods may be which architectural information is empirically captured, called in any order. could help. We are aware that identifying the SMs is a nontrivial task Step 2. Deriving ALTS: As shown, the ALTS generation in a development process not based on strictly formalized can be automated. In this work, we adapt the FC2Tools to refinement steps: In our experimentation with the TRMCS reach our goal, but other tools (such as Caesar/Aldebaran case study, this was not so difficult due to a straightforward [14]) can be used as well. correspondence between LTS labels and (reading and Step 3. Test Classes Selection: To cover the ALTS entails writing) operations performed over Java sockets. In Fig. 5, applying a coverage criterion over this graph, extracting the “s.inTo.readLong()” method represents the SM for only paths that expose different behaviors. Coverage Alarm. criteria, such as McCabe’s [40] test technique, which is Applying Step 4.3. At this point, we need to execute the used here, could be easily automated. test sequences and verify whether the system implemen- Step 4. Code-level Testing: This is the most difficult step tation behaves, at runtime, as described by our SA test. that the tester needs to apply to generate test cases. To trace As stated at the end of Section 4, to perform this analysis high-level information into code-level tests entails relating we apply a nondeterministic testing approach, instru- the abstract values of the specification to the concrete values menting the TRMCS program P to capture runtime of the implementation. Four general problems appear: information. Note that the instrumentation does not introduce new threads of execution, thus reducing 1. Understanding which classes and methods imple- instrumentation overheads. ment an architectural functionality is usually not More precisely, given the TRMCS implementation P , we obvious. Solving this problem may be facilitated by build an instrumented version P 0 able to store information using a more informative notation for LTSs, in which
  • 168 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 30, NO. 3, MARCH 2004 Inputs and Outputs are identified at the specification of these approaches use are the same as ours (e.g., LTS, level (IO Automata [37]) and can then be compared abstractions, event sequences), we use them in a different to code-level Inputs and Outputs. way. This difference in goals emerges from the very 2. More than one sequence can implement the desired beginning of our approach: We work on an architectural behavior. This problem can be handled only by description that drives our selection of the abstraction and requiring the intervention of the tester, who manu- of the paths. ally evaluates each point of decision and performs a Our approach to defining ALTS paths in order to specify reasoned choice regarding which selection is most high-level test classes has much in common with Carver relevant for testing purposes. and Tai’s use of the synchronization sequence (SYN- 3. When deriving the code-level sequences, we can end sequence) constraint for specification-based testing of up having to consider a lot of functionalities that are concurrent programs [16]. Indeed, sequencing constraints not described in the architectural sequences: Nota- specify restrictions to apply on the possible event sequences bly, in the SA description, we consider only of a concurrent program when selecting tests, similar to interactions, but not computations. what ALTS paths do for a SA. Again, the main difference is 4. The architectural model usually describes only the in the distance of the abstraction level of the reference expected behaviors, but the implementation has to model from the implementation. handle the exceptional behaviors as well. This prevents the straightforward application of their Currently, our approach is not entirely formal and a results for SA testing. For example, the technique defined in discontinuity between formality (Steps 1, 2, and 3) and [16] was implemented in [4] for Java programs, and the informality (Step 4) can be noticed. One way to cope with this SYN-sequences are runtime program executions; in our problem, is to make stronger assumptions on the process, approach, the SYN-sequences are the test classes and, thus, explicitly referring to a formal mapping between architec- they are more abstract. tural elements and implementation objects. This solution has A number of methods regard the selection of a test suite been investigated in [44], as outlined in Section 8. based on the specification of the implementation under test, which is assumed to be given in the form of a Finite 6.2 Approach Generality State Machine (FSM). The Transition tour, W-method, To put our approach into practice, we selected an ADL to DS-method, UIO method, and many variants of them (such describe the SA dynamics through an LTS, we selected a as the Wp-method) have been proposed. A review of these coverage criteria to extract a limited number of ALTS paths, approaches is given in [10]. Tan et al. present a test and we used an implementation following the object- generation tool (TAG) based on FSM and generated by a oriented paradigm. specification [51]. The drawbacks of this tool are due to the However, we kept our approach as independent as use of FSM with a limited number of states in the possible from these choices: The only constraint we implementation and the impossibility of ensuring a com- required is that the SA dynamics be described via an plete coverage with nondeterministic specifications. LTS conforming to the definition given in Section 2. Other approaches have been proposed to test software Different ADLs can be used to describe the SA. Different systems using statecharts: In [49], natural language ALTS coverage criteria can be applied and different scenarios are formalized through statecharts, annotated implementation languages and programming paradigms with helpful information for test case generation. Test cases can be used. Finally, several software development are then generated through path traversal. Bogdanov [11] processes can be used: The SA description can be obtained presents a method to extract test cases from statecharts in a posteriori in a reverse engineering step or the SA order to prove that the implementation is behaviorally description can be created before the system design, thus equivalent to the design. The Das-Boot tool presented in driving the definition of the components and of their [53] generates a test suite for object-oriented systems interactions or the SA description can be directly mapped specified through statecharts. on the implementation [41]. Several authors [25], [52], [22] have recently dealt with the problem of automatically generating test suites to test the conformance of an implemented system to its specifica- 7 RELATED WORK tion. Fujiwara and Bochmann et al. [25] and Petrenko [10] The approach we have proposed here makes use of use FSMs to drive a conformance testing approach: The several concepts: testing concurrent and real-time sys- system specification (S) and its implementation (I) are tems, specification-based testing, conformance testing, formalized using FSMs; “conformance relations” (equiva- tracing information, and architectural testing. This section lence, quasi-equivalence, reduction) are defined and the is a brief, but, for reasons of space, incomplete, overview Wp-method is used to generate test sequences. of these research areas. Tretmans [52] presents some approaches for LTS-based Much work has been devoted to testing concurrent and conformance testing. The specification (S) is formalized real-time systems, both specification-driven [16], [39], [15] using LTSs, the implementation (I) is expressed using LTSs and implementation-based [29], [4], [17], [35]. These or Input/Output Transition Systems (IOTSs), and tests (T ) references address different aspects, from modeling time are formalized using LTSs. Given S and one of its possible to internal nondeterminism, but all focus on unit testing. implementations I, Tretmans defines some “implementa- Our aim is different: We want to derive test plans for tion relations” (imp) to correlate S with I: I conforms to S integration testing. Thus, although the technical tools some iff I is “imp” with respect to S.
  • MUCCINI ET AL.: USING SOFTWARE ARCHITECTURE FOR CODE TESTING 169 Fernandez et al. [22], [23] use Input/Output LTS (IOLTS) SA [24], very little has been proposed for SA-based testing. to formalize the specification, the implementation, and the Despite the claim that SA must play a role throughout the test purposes. They propose an approach for an automatic, software life cycle, there are still many problems due to the on-the-fly generation of test cases, embodied in the Test practical implications of using SA models and tools. In Generation and Verification (TGV) environment [14]. particular, while it may be relatively easy to conceive of a We also use the SA-derived LTS as a reference model to method for deriving suites of architectural test cases derive test cases but in a different manner. The behavioral according to some notion of coverage, subsequently estab- model of the implementation under test could be modeled lishing a relationship between these high-level tests and the through an LTS (or an IOLTS). This model is required in implementation under test may be quite difficult and, as order to properly define what conformance means in a yet, no complete solution exists. formal manner. Based on these models, they define some As far as specification-based testing is concerned, implementation relations (conf, ioconf, ioco, etc.) and they although the methods and tools used basically remain the test whether the implementation is correct with respect to same, there are at least two main differences between our the specification. In our approach, we do not assume that approach and existing approaches: we are able to produce an LTS over the implementation. In fact, we compare architecture level sequence of events with 1. The SA description tries to capture SA-relevant lower level execution paths. behaviors alone, while abstracting away other The problem of tracing information is not new, and system functions. Thus, our tests specifically belong some relevant papers are listed in the references of the to the integration testing stages and certainly do not following sources. Egyed [20] shows a way to detect aim to test the system as completely as possible, as in traceability between software systems and their models traditional specification-based test approaches. and gives a list of interesting references on traceability 2. The abstraction level of the reference model and its techniques. Some work has been done in bridging the gap relative “distance” from the implementation under between requirements and software architectures (e.g., [50], test may vary much in the two contexts. In SA-based [45]), and much other work addresses the traceability of approaches, this distance is purposely very high, requirements (e.g., [26]). The problem of mapping abstract whereas, in existing approaches to specification- tests into the System Under Test is currently being studied based testing, this is often assumed to be low. In in the AGEDIS project [1]. other words, a high abstraction level is a basic The topic of architectural testing has recently raised assumption of our approach, whereas traditional some interest [47], [7], [27], [28], [48]. In [47], the authors approaches require a close distance between the define six architectural-based testing criteria, adapting reference model and the implementation, and specification-based approaches. In [7], the authors analyze typically both are represented by automata [52]. the advantages in using SA-level testing for reuse of tests We have provided here an excerpt from the empirical and to test extra-functional properties. In [27], Harrold insights gained while performing SA-based testing on the presents approaches for using software architecture for TRMCS case study. Although the experience reported may effective regression testing, and, in [28], she also discusses appear to be specific to the case study, we believe that, in the use of software architecture for testing. In [48], the most cases, the problems encountered as well as the solutions authors present an architecture-based integration testing found can easily be generalized to any SA-centered context. approach that takes into consideration architecture testabi- Our future work will go in several directions. We lity, simulation, and slicing. consider the approach presented here as a foundational All of the above works give interesting insights. contribution that might not be directly applied in practice. However, to the best of our knowledge, this paper For example, we might not always have a global architec- represents the first attempt to tackle the whole cycle of tural model at our disposal. This can happen for several SA-based testing with a comprehensive and concrete reasons: 1) architectural components may be described approach. It spans the spectrum from test derivation based through complex models, in terms of states and transitions on architecture dynamics down to test execution over and putting these models together may give rise to a state system implementation and relies on empirical hands-on explosion problem. 2) The architectural models may be experience derived from a real-world case study. incomplete, which means that some component behaviors are unknown or components are not completely specified. 8 CONCLUSIONS AND FUTURE WORK These are very common situations in industrial contexts. Our research investigates how the SA notions, methods, We are currently investigating the possibility of generating and tools can be usefully exploited within a software abstract observations and test cases directly from partial development process. In particular, this paper focuses on architectural models. using SA descriptions to improve the conformance testing Another area under investigation is how to further of a large distributed system. As such, this paper relies on formalize our approach, with stricter assumptions on the two prolific branches of current software engineering SA-based development process. In [44], we show how the literature: software architecture and specification-based testing process proposed in this paper can be made testing. completely systematic by specializing and refining our In terms of software architecture, although the literature general approach to a more specific context. By adopting a is rich in models and tools for design and analysis based on C2 style architecture [12] and the related C2 framework [13],
  • 170 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 30, NO. 3, MARCH 2004 we were able to handle the traceability/mapping among SA [21] FC2Tools, guide.html, Jan. 2004. and code problem (by adapting the category partition [22] J.-C. Fernandez, C. Jard, T. Jeron, L. Nedelka, and C. Viho, “Using method [46]) and the execution over the identified test cases On-the-Fly Verification Techniques for the Generation of Test Suites,” Proc. Eighth Int’l Conf. Computer Aided Verification, pp. 348- problems (through the Argus-I tool [2]). The approach has 359, 1996. been applied to a case study, gaining interesting results. [23] J.-C. Fernandez, C. Jard, T. Jeron, L. Nedelka, and C. Viho, “An Experiment in Automatic Generation of Test Suites for Protocols with Verification Technology,” Science of Computer Programming, ACKNOWLEDGMENTS vol. 29, pp. 123-146, 1997. [24] “Formal Methods for Software Architectures Tutorial Book on The authors would like to acknowledge the Italian MURST/ Software Architectures and Formal Methods,” Lecture Notes in Computer Science 2804, M. Bernardo and P. Inverardi, eds., 2003. MIUR national projects SALADIN and SAHARA, which [25] S. Fujiwara and G.v. Bochmann, “Test Selection Based on Finite partially supported this work as well as Thierry Jeron and State Models,” IEEE Trans. Software Eng., vol. 17, no. 6, pp. 591- the anonymous reviewers for their constructive comments 603, June 1991. [26] O.C.Z. Gotel and A.C.W. Finkelstein, “An Analysis of the and suggestions. Requirements Traceability Problem,” Proc. IEEE First Int’l Conf. Requirements Eng., pp. 94-102, Apr. 1994. [27] M.J. Harrold, “Architecture-Based Regression Testing of Evolving REFERENCES Systems,” Proc. Int’l Workshop the Role of Software Architecture in Testing and Analysis, pp. 73-77, July 1998. [1] AGEDIS Project, “Automated Generation and Execution of Test Suites for Distributed Component-Based Software,” http:// [28] M.J. Harrold, “Testing: A Roadmap,” Proc. ACM ICSE 2000 Conf.,, Jan. 2004. The Future of Software Eng., A. Finkelstein, ed., pp. 61-72, 2000. [2] The Argus-I project, Univ. of California, Irvine, http://www. [29] D.M. Hoffman and P.A. Stropper, “Techniques and Tools for Java, Jan. 2004. API Testing,” Proc. IEEE 2000 Australian Software Eng. Conf., [3] L. Bass, P. Clements, and R. Kazman, Software Architecture in pp. 235-245, 2000. Practice. Addison-Wesley, 1998. [30] C. Hofmeister, R.L. Nord, and D. Soni, Applied Software Archi- [4] A. Bechini and K.-C. Tai, “Design of a Toolset for Dynamic tecture. Addison Wesley, 1999. Analysis of Concurrent Java Programs,” Proc. IEEE Sixth Int’l [31] P. Inverardi and A.L. Wolf, “Formal Specifications and Analysis of Workshop Program Comprehension, pp. 190-197, June 1998. Software Architectures Using the Chemical Abstract Machine [5] G. Bernot, M.C. Gaudel, and B. Marre, “Software Testing Based on Model,” IEEE Trans. Software Eng., vol. 21, no. 4, pp. 100-114, Apr. Formal Specifications: A Theory and a Tool,” Software Eng. J., 1995. vol. 6, no. 6, pp. 387-405, 1991. [32] P. Inverardi and H. Muccini, “The Teleservices and Remote [6] A. Bertolino, “Knowledge Area Description of Software Testing,” Medical Care System (TRMCS),” IEEE Proc. Int’l Workshop Software SWEBOK, Joint IEEE-ACM Software Eng. Coordinating Committee, Specification and Design, Nov. 2000., 2000. [33] R. Kazman, L. Bass, G. Abowd, and M. Web, “SAAM: A Method [7] A. Bertolino and P. Inverardi, “Architecture-Based Software for Analyzing the Properties of Software Architectures,” Proc. Int’l Testing,” Proc. Int’l Software Architecture Workshop, Oct. 1996. Conf. Software Eng., pp. 81-90, May 1994. [8] A. Bertolino, F. Corradini, P. Inverardi, and H. Muccini, “Deriving [34] Labelled Transition System Analyzer (LTSA), http://www-dse. Test Plans from Architectural Descriptions,” ACM Proc. Int’l Conf., Jan. 2004. Software Eng., pp. 220-229, June 2000. [35] B. Long, D. Hoffman, and P. Strooper, “Tool Support for Testing [9] A. Bertolino, P. Inverardi, and H. Muccini, “An Explorative Concurrent Java Components,” IEEE Trans. Software Eng., vol. 29, Journey from Architectural Tests Definition downto Code Tets no. 6, pp. 555-566, June 2003. Execution,” IEEE Proc. Int’l Conf. Software Eng., pp. 211-220, May [36] D.C. Luckham, J.J. Kenney, L.M. Augustin, J. Vera, D. Bryan, and 2001. W. Mann, “Specification and Analysis of System Architecture [10] G.v. Bochmann and A. Petrenko, “Protocol Testing: Review of Using Rapide,” IEEE Trans. Software Eng., vol. 21, no. 4, pp. 336- Methods and Relevance for Software Testing,” ACM Proc. Int’l 355, Apr. 1995. Symp. Software Testing and Analysis, pp. 109-124, 1994. [37] N.A. Lynch and M.R. Tuttle, “An Introduction to Input/Output [11] K. Bogdanov, “Automated Testing of Harel’s Statecharts,” PhD Automata,” CWI Quarterly, vol. 2, no. 3, pp. 219-246, Sept. 1989. thesis, The Univ. of Sheffield, 2000. [38] J. Magee and J. Kramer, Concurrency: State Models & Java Programs. [12] The C2 style and ADL Project web page, Wiley, Apr. 1999. architecture/c2.html, Jan. 2004. [39] D. Mandrioli, S. Morasca, and A. Morzenti, “Generating Test [13] The C2 Framework, Cases for Real-Time Systems from Logic Specifications,” ACM software.html, Jan. 2004. Trans. Computer Systems, vol. 13, no. 4, pp. 365-398, Nov. 1995. [14] Caesar/Aldebaran Tool (CADP),, [40] T.J. McCabe, “A Complexity Measure,” IEEE Trans. Software Eng., Jan. 2004. vol. 2, no. 4, pp. 308-320, 1976. [15] R. Cardell-Oliver and T. Glover, “A Practical and Complete Algorithm for Testing Real-Time Systems,” Proc. Fifth Int’l School [41] N. Medvidovic, D.S. Rosenblum, and R.N. Taylor, “A Language and Symp. Formal Techniques in Real-Time and Fault-Tolerant and Environment for Architecture-Based Software Development Systems, A.P. Ravn and H. Rischel, eds., pp. 251-261, Sept. 1998. and Evaluation,” Proc. IEEE Int’l Conf. Software Eng., pp. 44-53, [16] R.H. Carver and K.-C. Tai, “Use of Sequencing Constraints for 1999. Specification-Based Testing of Concurrent Programs,” IEEE Trans. [42] R. Milner, Communication on Concurrences. Int’l Series on Compu- Software Eng., vol. 24, no. 6, pp. 471-490, June 1998. ter Science, Prentice Hall, 1989. [17] J. Choi and H. Srinivasan, “Deterministic Replay of Java Multi- [43] H. Muccini, A. Bertolino, and P. Inverardi, “Using Software threaded Applications,” Proc. ACM Sigmetrics Symp. Parallel and Architecture for Code Testing,” Technical Report TRCS 006/2004, Distributed Tools, pp. 48-59, 1998. ´ Dipartimento di Informatica, Universita degli Studi di L’Aquila, [18] E.M. Dashofy, A. van der Hoek, and R.N. Taylor, “An, Infrastructure for the Rapid Development of XML-Based Archi- 2004. tecture Description Languages,” Proc. 24th Int’l Conf. Software Eng., [44] H. Muccini, M. Dias, and D.J. Richardson, “Systematic Testing of 2002. Software Architectures in the C2 Style,” Proc. Int’l Conf. Funda- [19] J. Dick and A. Faivre, “Automating the Generation and Sequen- mental Approaches to Software Engineering (FASE 2004), 2004. cing of Test Cases from Model-Based Specifications,” Proc. [45] B. Nuseibeh, “Weaving Together Requirements and Architec- FME’93 Conf.: Industrial-Strength Formal Methods, J.C.P. Woodcock tures,” Computer, vol. 34, no. 3, pp. 115-117, Mar. 2001. and P.G. Larsen, eds., pp. 268-284, 1993. [46] T.J. Ostrand and M.J. Balcer, “The Category-Partition Method for [20] A. Egyed, “A Scenario-Driven Approach to Traceability,” Proc. Specifying and Generating Functional Tests,” Comm. ACM, vol. 31, IEEE Int’l Conf. Software Eng., pp. 123-132, May 2001. no. 6, pp. 676-686, June 1988.
  • MUCCINI ET AL.: USING SOFTWARE ARCHITECTURE FOR CODE TESTING 171 [47] D.J. Richardson and A.L. Wolf, “Software Testing at the Antonia Bertolino is a researcher at the Architectural Level,” Proc. Software Architecture Workshop and Proc. Institute of Information Science and Technolo- ACM SIGSOFT ’96 Workshop, pp. 68-71, 1996. gies (ISTI) of the Italian National Research [48] D.J. Richardson, J. Stafford, and A.L. Wolf, “A Formal Approach Council (CNR) in Pisa, Italy, where she leads to Architecture-Based Software Testing,” technical report, Univ. of the Software Engineering Group and the Pisatel California, Irvine, 1998. Laboratory. Her research interests are in soft- [49] J. Ryser and M. Glinz, “A Practical Approach to Validating and ware engineering, especially software testing Testing Software Systems Using Scenarios,” Proc. Third Int’l and dependability. Currently, she is investigating Software Quality Week Europe Conf., Nov. 1999. approaches for systematic integration test stra- [50] J. Castro and J. Kramer, “From Software Requirements to tegies, for architecture and UML-based test Architectures,” Proc. Straw ’01 First Int’l Workshop, May 2001. approaches, and for component-based software analysis and testing. [51] Q.M. Tan, A. Petrenko, and G.v. Bochmann, “A Test Generation She is an associate editor of the Journal of Systems and Software, of Tool for Specifications in the Form of State Machines,” Proc. IEEE Empirical Software Engineering Journal, and of the IEEE Transactions Int’l Comm. Conf., pp. 225-229, June 1996. on Software Engineering. She was the general chair of the ACM [52] J. Tretmans, “Conformance Testing with Labeled Transition Symposium on Software Testing and Analysis (ISSTA 2002) and of the Systems: Implementation Relations and Test Generation,” Com- Second International Conference on Achieving Quality in Software puter Networks and ISDN Systems, vol. 29, pp. 49-79, 1996. (AquIS ’93). She has served on the program committees of several [53] M. Vieira, M. Dias, and D.J. Richardson, “Object-Oriented conferences and symposia, including ISSTA, Joint ESEC-FSE, ICSE, Specification-Based Testing Using UML Statechart Diagrams,” SEKE, Safecomp, and Quality Week. She has (co)authored more than Proc. Int’l Conf. Software Eng. 2000 Workshop Automated Program 60 papers in international journals and conferences. Analysis, Testing, and Verification, pp. 758-761, June 2000. [54] A.H. Watson and T.J. McCabe, “Structured Testing: A Testing Paola Inverardi is a full professor at University Methodology Using the Cyclomatic Complexity Metric,” Nat’l of L’Aquila. Previously, she worked at IEI-CNR Inst. of Standards and Technology Special Publication 500-235, in Pisa and at the Olivetti Research Laboratory Aug. 1996. in Pisa. She is the head of the Department of [55] M. Young, “Testing Complex Architectural Conformance Rela- Computer Science at the University of L’Aquila, tions,” Proc. Int’l Workshop Role of Software Architecture in Testing where she leads the Software Engineering and and Analysis, pp. 42-45, July 1998. Architecture Research Group. Her main re- search area is in the application of formal Henry Muccini has been an assistant professor methods to software development. Her research in the Computer Science Department at the interests primarily concentrate in the field of University of L’Aquila (Italy) since 2002 and he software architectures. She has actively worked on the verification and has been a visiting professor in the Information analysis of software architecture properties, both behavioral and and Computer Science Department at the quantitative, for component-based, distributed and mobile systems. University of California, Irvine. His research She has served as general chair, program chair, and program interests are in software architecture-based committee member for many international conferences. She is a analysis techniques, specifically, in testing of member of the IEEE. software subsystems integration against a soft- ware architecture specification, in product line architecture specification and testing, in integrating coordination policies . For more information on this or any other computing topic, inside the architectural description and validating software architectures please visit our Digital Library at using semiformal specifications. Recently, he has also been investigat- ing how Web applications may be modeled using UML formalisms and through product line analysis.