Regression Testing of COTS Component-Based Software System

1,799 views

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,799
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
76
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Regression Testing of COTS Component-Based Software System

  1. 1. Regression Testing of COTS Component-Based Software System A Thesis Submitted in partial fulfillment of the requirement for the Award of the Degree of Master of Technology IN COMPUTER SCIENCE & ENGINEERING ((22000066--22000088)) Under the Supervision of Ms. Gagandeep SSuubbmmiitttteedd BByy YYooggeesshh KKuummaarr DDeeppaarrttmmeenntt ooff CCoommppuutteerr SScciieennccee PPuunnjjaabbii UUnniivveerrssiittyy,, PPaattiiaallaa.. 1
  2. 2. Acknowledgment While this report may bear only one name on the cover, it is the result of inputs from others who have helped me during the thesis work. In an effort to recognize the contributions of those who helped bring thesis to life, I would like to acknowledge and thank them. First and foremost, I sincerely thank my project supervisor, Ms. Gagandeep, for her continuous guidance with the thesis work. Without her encouragement, this thesis would not have been possible. I would also like to thank Dr. Gurpreet Singh Lehel, Head, Department of Computer Science, for his help throughout. I am also thankful to the Department of Computer Science, because I have utilized the resources available here for research purposes, and this has enormously helped me throughout the duration of this thesis. And last but not least, I wish to express my eternal gratitude towards my family for their everlasting support. I am greatly indebted to them. Yogesh Kumar 2
  3. 3. Abstract The primary role of component-based software engineering is to address the development of systems as an assembly of parts (components), the development of parts as reusable entities, and the maintenance and upgrading of systems by customizing and replacing such parts. This requires established methodologies and tool support covering the entire component and system lifecycle including technological, organizational, marketing, legal, and other aspects. The traditional disciplines from software engineering need new methodologies to support component -based development. Component-based Software Engineering (CBSE) has become recognized as such a new sub-discipline of Software Engineering. Component based software engineering (CBSE) also offers convenience as well as fast design/ implementation. Testing UML design models is challenging because a design model consists of many models that describe different design views. An approach to generating inputs for testing UML designs requires a mechanism that combines information from both the structural and behavioral views of a design. Regression testing is a testing process which is applied after a program is modified. It involves testing the modified program with the old test cases in order to re-establish confidence in the program. As a part of our research work, we have proposed a methodology to generate regression test suite based on UML sequence diagrams. We have constructed and implemented control flow graph (CFG) in C++ for the storage of UML sequence diagrams. The sequence diagrams have been drawn with the help of NET BEANS tool. We have practically experimented on different component from different application domain. We have stored the sequence diagrams for each component into the control flow graph (CFG) and traversed it for the generation of the regression test suite. The results have been evaluated by calculating the total running time for each component. Our technique of automating the regression test suite generation in effective in terms of reduction in number of test cases and time efficient as compared to manual technique. 3
  4. 4. . TABLE OF CONTENTS 1. Introduction 1 1.1 Problem …………………………………………………………1 1.2 Overview of the solution and research challenges………………2 Scope of the research…………………..………………………...3 1.3 Thesis organization………………………………………………3-4 2. Regression Testing …………………………………………………5 2.1 Need for regression testing……………………………………….5 2.2 Types of regression testing……………………………………….6 2.3 Difference between testing and regression testing………………..7 2.4 Similarities between testing and regression testing……………....8 2.5 Problem of regression testing…………………………………….9 2.6 Retest approach…………………………………………………..9-13 3. Literature Survey………………………………………………….14 3.1 Techniques of regression testing………………………………..14-16 4. Case studies of the existing techniques …………………………. 18 4.1 Generating regression tests via model checking………………...18-26 4.1 Regression testing UML design…………………………………27-36 5. Related work in component-based Regression testing and UML.37-39 6. Control flow graph based criteria to generate regression test suite…………………………………………………………….40-46 7. Case Study …………………………………………………………..47 7.1 Case study with ATM using Net Beans Tools for UML sequence diagram…………………………………………………………....47-51 4
  5. 5. 7.2 Case study with E-Ticket of Cinema for UML sequence diagram..52-56 7.3 Case study with Library Record System for UML sequence diagram……………………………………………………………57-61 8. Conclusion and Future work ………………………………………62 8.1 Evaluation of the case studies……………………………………62-64 8.2 Summary of the contribution………………………………………65 8.3 Future work…………………………………………………….......65 Reference……………………………………………………………….66-69 5
  6. 6. LIST OF TABLE Table 4.1.1 Test suite of the window 98 notepad………………………19 Table 4.1.2 Test suite of window XP notepad………………………….20 Table 4.1.3 Test suite of comparator…………………………………...23 Table 4.1.4 Test suite of coverage selector ……………………………24-25 Table 4.1.5 Test suite of test generator………………………………...25-26 Table 4.2.1 Test suit T of the Program P (simple program of factorial)..30 Table 4.2.2 Test suit T’ of the Program P’ (modified program of factorial)..………………………………………………….32-33 Table 4.2.3 UML Test suits of combined of T and T’…………………33-34 Table 4.2.4 Final Test cases …………………………………………35-36 Table 7.1.1 Case scenarios based on the ATM sequence diagram of first version……………………………………………………...48 Table 7.1.2 Case scenarios based on the ATM sequence diagram of modified version …………………………………………...50 Table 7.1.3 Final Regression Test case generation from the ATM Component ………………………………………………….51 Table 7.2.1 Case scenarios of first version of the E-Ticket system of Cinema.. …………………………………………………..52-53 6
  7. 7. Table 7.2.2 Case scenarios of modified version of the E-Ticket system of Cinema ……………………………………………………54-55 Table 7.2.3 Final Regression Test case generation from the E-Ticket System of Cinema Component……………………………………..55-56 Table 7.3.1 Case scenarios of the first version of the Library Record System component……………………………………………58 Table 7.3.2 Case scenarios of the modified version of the Library Record System component …………………….………….59-60 Table 7.3.3 Final regression test suite for Library record system……...60-61 7
  8. 8. LIST OF FIGURES Figure 2.1 Regression testing ……………………………………………6 Figure 2.2 The retest regression testing process for corrective regression testing………............................................................................11 Figure 2.3 An operational overview of Retest…………………................12 Figure 4.1.1 Functionalities of comparator…………….….........................22 Figure 4.1.2 Coverage Selector……………………………........................24 Figure 4.2.1 Object Method Directed Acyclic Graph (OMDAG) for simple program of factorial ...............................................29 Figure 4.2.2 Object Method Directed Acyclic Graph (OMDAG) for modified version of factorial………………………….....31 Figure 6.1 Working of the control flow graph (CFG) based criteria for generation of regression test suite…………………………......40 Figure 6.2 Sequence diagram of student enrollment system……………...41 Figure 7.1.1 First version of the ATM Sequence Diagram………………..47 Figure 7.1.2 Extending the functionality of first version of ATM sequence Diagram…………………………………………………….49 Figure 7.2.1 Sequence diagram of first version of E-Ticket of Cinema component…………………………………………52 Figure 7.2.2 shows the sequence diagram of modified version of E-Ticket of Cinema……………………………………………………….54 8
  9. 9. Figure 7.3.1 Sequence diagram of the first version for the Library Record System………………………………………………………..57 Figure 7.3.2 Sequence diagram of the modified version for the Library Record system………………………………………………..59 Figure 8.1.1 Bar chart of the total number of regression test suite for ATM Component…………………………………………………….62 Figure 8.1.2 Bar chart of the total number of regression test suite for E- Ticket System for Cinema Component………………………63 Figure 8.1.3 Bar chart of the total number of regression test suite for Library Record System Component…………………………………..64 9
  10. 10. Chapter 1 Introduction 1.1 Problem In recent years, Component-based software has been widely used in various application domains and becomes a fairly popular software form. However, Due to the lack of information about the externally developed components, system testers (i.e., component users) generally can't perform effective testing (especially regression testing) on their component based systems. Component users don't know the details about change in component, so they aren't able to select the proper test cases to retest the modified system [3]. Meanwhile, Component-Based Software Engineering (CBSE) is a continuously growing field. However, there is a lack for good environment support for the CBSE activities, mainly because most of the environments do not offer comprehensive, integrated support for the full range of CBSE activities. The construction of this kind of environments involves general environment issues, such as tool integration, and specific CBSE issues, such as domain engineering and application composition. The goal of component-based software engineering is to increase the productivity, quality [3]. In component-based software systems, regression test case selection is a challenge since vendor of commercial-off-the-shelf (COTS) component does not release source code. Regression testing is applied to modified software to provide confidence that modified code behaves as intended, and does not adversely affect the behavior of unmodified code. Regression testing plays an integral role in software maintenance; without proper regression testing we are reluctant to release modified software [7]. One characteristic distinguishing regression testing from developmental testing is the availability, at regression test time, of existing test suites. If we reuse such test suites to retest a modified program, we can reduce the effort required to perform that testing. Unfortunately, test suites can be large, and we may not have time to rerun all tests in such suites. Thus, we must often restrict our efforts to a subset of the previously existing tests. We call the problem of choosing an appropriate subset of an existing test suite the 10
  11. 11. selective retest problem; we call a method for solving this problem a selective retest method [18]. We use the strategy for regression test selection in component based software engineering. This is achieved by the component developer providing information about the changes in the first one version to next and the component user utilizing this information in regression test case selection. We use Unified Modeling Language (UML) and its sequence diagram to formally specify the test case selection process. UML is the accepted standard for component design, and by specifying the regression test case selection process [19]. 1.2 Overview of the Solution and the research challenges Improvements in software testing techniques are needed to address the increasingly complex applications that are implemented by today’s software systems. The UML and its diagrams are widely used to visually depict the static structure and more importantly for us, the dynamic behavior of such applications. This trend provides us with an excellent opportunity to meld our proven test generation technology (TDE) with the UML and give developers and testers the capability of automatically generating black- box conformance tests early on. For regression testing, we can derive tests from UML Sequence diagrams, which embody the behavioral description of each component of interest and more importantly, the interactions or events between them. The Unified Modeling Language (UML) [14] reduces the efforts in constructing the models in software development and increases maintainability and reusability during the life cycle. Another well accepted method is provided through design patterns. Likewise, frameworks provide users with an efficient means to combine component reuse with design reuse. Component based software engineering (CBSE) also offers convenience as well as fast design/ implementation. Testing UML design models is challenging because a design model consists of many models that describe different design views. An approach to generating inputs for testing UML designs requires a mechanism that combines information from both the structural and behavioral views of a design. We conducted the case studies to investigate the effectiveness of our testing approach[3]. 11
  12. 12. The studies include the effective way of generating the UML regression test case selection based on UML sequence diagram. In my thesis, I have generated the storage graph for sequence diagram using C++ program of data structure. The studies demonstrate the cost of testing in terms of number of test inputs and effectiveness of test case generation in terms of execution time. Scope of the Research The work described in my thesis focuses on effective way of generating the UML regression test suite with the help of control flow graph (CFG) for the storage of the sequence diagram. And the traversal of the control flow graph will automatically produce the final regression test suite. The practical approach for generating the regression test suite will provide more accuracy as compared to the manually generating the test suite. The control flow graph for regression test selection techniques attempt to reduce the time required to retest a modified program by selecting some subset of the existing test suite. The purpose of RTS techniques is to reduce the high cost of retest-all regression testing by selecting a subset of possible test cases. In our approach, a control flow graph (CFG) will store the each events of the sequence diagram. They store the complete methods with parameters and information about the sender and the receiver and with their pre-conditions and post-conditions. The specifications consist of pre- and post-conditions expressed in the OCL. 1.3 Project Organization The rest of the thesis is organized as follows: • Chapter 2 describe the Regression testing and its types. • Chapter 3 presents Literature Survey of regression testing and defines the various techniques of the regression testing. • Chapter 4 describes the case studies of the existing Techniques. • Chapter 5 discusses related work in component based regression testing and UML. • Chapter 6 Control flow graph (CFG) based criteria to generate regression test suite. 12
  13. 13. • Chapter 7 describes our technique for generating the final regression test suite from UML sequence diagram from different software component. • Chapter 8 Conclusion and Future work. 13
  14. 14. Chapter 2 Regression Testing Regression testing is a testing process, which is applied after a program, is modified. It involves testing the modified program with some test cases in order to re-establish our confidence that the program will perform according to the (possibly modified) specification. In the development phase, regression testing may begin after the detection and correction of errors in a tested program. A tested program is a program which has been tested with a high quality test plan. Regression testing is a major component in the maintenance phase where the software system may be corrected, adapted to new environment, or enhanced to improve its performance. Modifying a program involves creating new logic to correct an error or to implement a change and incorporating that logic into an existing program. The new logic may involve minor modifications such as adding, deleting, rewriting a few lines of code, or major modifications such as adding, deleting [9]. 2.1 The Need for Regression Testing Software inevitably changes however well conceived and well written it initially may be. Operational failures expose faults to be repaired. Mistaken and changed requirements cause the software to be reworked. New uses of old software yield new functionality not originally conceived in the requirements. The management of this change is critical to the continuing usefulness of the software. The new functionality added to a system may be accommodated by the standard software development pro- cases. Regression testing [9] attempts to revalidate the old functionality inherited from the old version. The new version should behave exactly as the old except where new behavior is intended. Therefore, regression tests for a system may be viewed as partial operational requirements for new versions of the system. Figure1 shows a typical example of a sequence of time intervals during the life of a software system. 14
  15. 15. Figure 2.1 Regression Testing 2.2 Types of Regression Testing Two types of regression testing can be identified based on the possible modification of the specification. Progressive regression testing involves a modified specification. Whenever new enhancements or new data requirements are incorporated in a system, the specification will be modified to reflect these additions. In most cases, new modules will be added to the software system with the consequence that the regression testing process involves testing a modified program against a modified specification [7]. In corrective regression testing, the specification does not change. Only some instructions of the program and possibly some design decisions are modified. This has important implications because most test cases in the previous test plan are likely to be valid in the sense that they correctly specify the input-output relation. However, because of possible modifications to the control and data flow structures of the software, some existing test cases are no longer testing the previously targeted program constructs. The corrective regression testing is often done after some corrective action is performed on the software. 15
  16. 16. 2.3 Differences between Testing and Regression Testing Most people assume that regression testing is a simple extension of testing. However, it is not always the case. There are several major differences between these two processes. Availability of test plan Testing begins with a specification, an implementation of the specification and a test plan with test cases added during the specification, design and coding phases. All these test cases are new in the sense that they have not been used to exercise the program previously. Regression testing starts with a possibly modified specification, a modified program and an old test plan which requires updating. All test cases in the test plan were previously run and were useful in testing the program. Scope of test The testing process aims to check the correctness of a program, including its individual components (e.g., functions and procedures) and the interworking of these components. Regression testing is concerned checking the correctness of parts of a program. The portion of a program which is not affected by modifications need not be retested. An interesting problem is to determine the affected portion of a program in an efficient manner. A solution to this problem may be the use of retestable units. Time allocation Testing time is normally budgeted before the development of a product. This time can be as high as half the total product completion time. However, regression testing time, especially time for corrective regression testing, is not normally included in the total product cost and schedule. Consequently, when regression testing is done, it is nearly always performed in a crisis situation. The tester is normally urged to complete retesting as soon as possible and most often is given limited time to retest. Development information In testing, knowledge about the software development process is readily available. In fact, the testing group and the development group may be the same. Even if an organization has a separate testing group, the testers can usually query the developers about any uncertainty in the software. But in regression testing, the testers most likely will not be the developers of the product. Since regression testing may be done at a different time and place, the original developers may no longer be available. This situation suggests that any relevant development information should be retained if regression testing is to be successful. 16
  17. 17. Completion time The completion time for regression testing should normally be less than that for testing since only parts of a program are being tested. 2.4 Similarities between Testing and Regression Testing Several aspects of regression testing are similar to that of testing. In particular, the purposes and testing techniques used are almost the same. Purposes: The purposes of testing and regression testing are quite similar. They both aim to: 1. Increase one’s confidence in the correctness of a program. 2. Locate errors in a program. Some additional goals of regression testing are to: preserve the quality of the software; the modified software should be at least as reliable as its previous version; this may be achieved in many ways; one possible method is to insist that the same structural coverage is achieved by both versions of the software; ensure the continued operation of the software; this is an important goal because some users may become dependent on the software product, and software developers have a responsibility to continue to provide the same service to users. Testing techniques Since test cases in a test plan depend on the chosen testing technique, the testing technique used by both testing and regression testing should be the same if the regression testing process involves the reuse of test cases. If regression testing were to involve a different testing technique, then it would be difficult to reuse the existing test plan. Another reason for using the same testing technique is that it is easier to evaluate the quality of two software products if they are tested by the same technique. At the current state of the art, it is difficult to compare the relative test effectiveness of two different testing techniques [7]. 17
  18. 18. 2.5 The Problem of Regression Testing The problem of regression testing [8] may be broken down into two subproblems: the test selection problem and the test plan update problem. The test selection problem is concerned with the design and selection of test cases to fully test a modified program. Some test cases may be selected from those in the existing test plan, while others have to be created based on modifications made to the program. The test plan update problem deals with the management of a test plan as a program is undergoing successive modifications. Certain test cases will become obsolete and new test cases must be added to test the modified and new features of the software. The test plan update problem may be defined as follows: Test Plan Update Problem: Given a program P, and its specification S, a test plan T for testing P, a program P’ which is a modified version of P, and its specification S’, generate a test plan T’ for P’ from T, P, S, F” and S’. From this definition, we can break down the test plan update problem into two subproblems: 1. Test case classification problem given a program P, and its specification S, a test plan T for testing P, a program P’ which is a modified version of P, and its specification S’, group the test cases in T into three mutually exclusive sets: reusable, retestable, and obsolete. 2. Update the test plan by deleting the obsolete test cases, and adding in the new- structural and new-specification test cases. Observe that before we can solve subproblem (2), we (2) Update problem need a solution to the test selection problem [15]. 2.6 The Retest Approach In this section, we describe an approach for corrective regression testing. The Retest approach [9] tries to reuse the test plan in a way that will reduce the amount of testing. The previous test plan is used to assist in test selection, and is analyzed in order to classify the various test cases; only a subset of the previous test plan is rerun. The Retest regression testing process consists of two major phases: the test case classification phase 18
  19. 19. and the test plan update phase (see following Figure2.2). We next describe the major components of this regression testing process. Static Analyzer This component is static in the sense that no test case is executed. All analysis is done using the change information and the information stored in the test plan. The static analyzer uses this information to classify the existing test cases into two classes: reusable and unclassified. The reusable test cases are those tests which do not exercise any modified code, while the unclassified test cases are those which do. Another function of the static analyzer is to compute the structural coverage measure achievable from the reusable test cases based on the stored dynamic behavior of these test cases. Both the coverage measure and the unclassified test cases are passed to the Dynamic Analyzer. Dynamic Analyzer This component is dynamic because it actually executes some test cases in order to: 1) Test the program, 2) Satisfy the structural coverage criterion, and 3) Classify the unclassified tests into obsolete and retestable test classes. Figure 2.2 the Retest regression testing process for Corrective Regression Testing 19
  20. 20. The Dynamic Analyzer executes each test which is unclassified in an order implied by the test-case-order. These test cases will be executed according to their ordering within the test-case-order. Note that all specification-based tests will be run first and put in the retestable class. They will not become obsolete because the specification is assumed to be unchanged. Any structural test which does not increase the structural coverage measure will be put into the obsolete test class. All others are grouped into the retest able test class. The Dynamic Analyzer will stop test case execution whenever the structural coverage criterion is satisfied. Any remaining unclassified tests may be placed into the obsolete class, or handled according to the testing guideline specified in the test an output of the Dynamic Analyzer is the coverage measure achieved by both the reusable and the retestable test cases. If this coverage measure is not satisfactory, then the Dynamic Tester is invoked; otherwise the testing process enters the test plan update phase. Dynamic Tester: This component is similar to the Dynamic Analyzer and is invoked only when the structural coverage criterion is not satisfied by both the reusable and retestable test cases. A major function of this component is to execute the new structural test cases which are designed to exercise the modified code or to satisfy the structural coverage criterion. Only the test cases which will increase the structural coverage measure will be stored. Once the structural coverage criterion is satisfied, the Test Plan Updater is invoked [9] 20
  21. 21. Figure 2.3 an operational overview of Retest A major difference between the Dynamic Analyzer and the Dynamic Tester is that the former involves the changing and deleting of existing information in the test plan, while the latter involves the execution of new tests and the addition of new information to the test plan. There are two advantages in running the unclassified tests before running the new-structural test cases: • Some unclassified tests will test the modified code and therefore they can reduce the effort in designing new tests. • Some unclassified tests may be used to assist in test generation. 21
  22. 22. Test Plan Updater: This component creates a new test plan from the reusable, retestable and new-structural test cases. The objective is to generate an up-to-date test plan for the next cycle of modifications and regression testing. Most of the information stored in the new test plan have actually been collected by the Dynamic Analyzer and the Dynamic Tester. An operational overview of the Retest process is presented in above Figure 3.2. 22
  23. 23. Chapter 3 Literature Survey Regression testing aims at providing confidence that the modifications are correct and have not adversely affected other parts of the program. Regression testing can be progressive or corrective the former involves retesting major changes to the program’s specification. The latter is performed on a specification that essentially remains unchanged, so that only minor modifications, which do not affect the overall program structure, require retesting. For regression testing, it would be costly to repeat the whole set of test cases used in the initial development of the program and unreliable to choose a random subset of these test cases [4]. Therefore, it is important to select a suitable subset of test cases that accomplish the objectives of regression testing. The selection of suitable test cases can be made in different ways, and a number of regression testing approaches and algorithms have been proposed. A strategy based on input partitioning and cause-effect graphing of the program specification has been described. Leung and White have suggested classifying the initial test cases as reusable, retestable, obsolete, and adding new-structural and new- specification test cases and, then, selecting test cases from some of or all five classes. They have also proposed methods to limit integration regression testing to a small set of modules. 4.1 Techniques of Regression Testing: Gupta, Harrold, and Soffa’s proposed slicing algorithm which is data flow based. It uses slicing to explicitly detect definition-use pairs that are directly or indirectly affected by a program modification. A program slice consists of all statement segments, including conditionals that might affect the values of variables in the modified statement/segment. The advantage of this slicing approach is that no data flow history is needed, nor is the recomputation of data flow for the entire program required after its modification [8]. Leung and White have proposed the concept of “firewall” to assist the tester in integration regression testing. For our study’s purpose, we have implemented the “firewall” concept at the segment (vs. module) level and hence the name adapted firewall algorithm. Leung and White suggest building “firewalls” to confine integration 23
  24. 24. regression testing to a small set of modules rather than allowing it to spread to many other modules. A “firewall” involves the modules that are modified and their direct ascendants and direct descendants [8]. An optimal retesting approach based on a 0-1 integer programming problem formulation has been proposed by Fischer and extended in and solved by natural optimization in Some software tools, which are based on some of the afore-mentioned techniques, have been constructed. Agrawal, Horgan, and Krauser’s incremental algorithm which selects test cases from T whose outputs may be affected by the modifications made to the program [8]. The incremental approach exploits the following observations: • Not all statements in the program are executed under all test cases. • If a statement is not executed under a test case, it can not affect the program output for that test case. Even if a statement is executed under a test case, it does not necessarily affect the program output for that test case. Every statement does not necessarily affect every part of he program output. These observations can be used to determine execution, dynamic, and then relevant slices for the test cases. Rothermel and Harrold divide regression testing into two phases for the purpose of cost analysis. During the preliminary phase, changes are made to the software, and the new version of the software is built. During the critical phase, the new version of the software is tested prior to its release to customers. They described regression testing techniques as follows: “Given a program P, a modified version P’, and a set T of test cases used previously to test P, regression analysis and testing techniques attempt to make use of a subset of T to gain sufficient confidence in the correctness of P’ with respect to behaviors from P retained in P’.” [14] Regression testing is an active area of research. Most of the work involves white box testing where code is available to determine the test cases. Agrawal et al. addressed the problem of determining test cases in the regression test suite on which new and old programs may produce different outputs [ 8] . They refer the problem of determining regression test suite on which the modified program may differ from the original program as the incremental regression test problem. 24
  25. 25. They proposed good approximate solution to this problem with execution slice technique to save regression testing effort with the focus on finding the test cases on which the original and the modified program differ. Wu et al. proposed a regression testing technique that selects test cases by utilizing static information from the analysis of the program structure and dynamic information by tracing the function-calling sequence. Their approach is based on data-flow method with optimization to handle OO features, such as inheritance, dynamic binding polymorphism and message passing. They need to create Functional Calling Graph (FCG) of each test case to trace the functional execution sequence. Harrold et al. Proposed the use of component meta-content or metadata to support regression testing of component-based software. Metadata stores additional information along with the components. In this sense, it is similar to the version information we require for components[8]. 25
  26. 26. Chapter 4 Case studies of the existing techniques 4.1 Generating Regression Tests via Model Checking Regression Testing is a maintenance activity that attempts to validate modified software and ensure that modifications are correct and have not inadvertently affected the software. Selective regression testing attempts to identify test cases for reuse in testing modified portions of the system. Specification-based regression testing techniques select reusable test cases from the original test suite with respect to formal specifications of the system. One should make sure that the cost of selecting the regression tests is lower than retesting with the entire original test suite. In this implementation, our work is to check the ability of model checkers to generate counterexamples, where each counterexample can be considered as a complete test Sequence. The general idea of our work is to use the difference between two versions of software specifications– the original one and the modified version. Our approach, therefore, supports different test coverage criteria. Currently, our work allows users to choose from a limited set of FSMbased test coverage criteria to generate regression test suites, including state transition level (STL), syntactic trigger events level (SynL) and semantic trigger events level (SemL). Component We are taking the example of GUI component. In this, we are comparing the functionalities of two version of GUI; first one is window 98 notepad and second is window XP notepad and possibly generates the test suit from these techniques. We follow the approach, RTMC (Regression Testing via Model Checking), which relies on the existence of two different specification models – Spec#2(XP notepad) is the specification of a modified version of the same system described by Spec#1(98 notepad). The idea is to use the difference between the two models to guide regression test case generation. 26
  27. 27. 98 notepad Spec1# Notepad of window 98 Specifications File Menu Edit Menu Select 1 New to open new file Cut cut the selected text Find Finds the specified text in a file and stop at first occurrence of text 2 Open to open existing file size should be less then 64kb Copy copy the selected text Find Next Finds the next occurrence of text entered in find box 3 Save Save current file Paste paste the text at cursor position 4 Save as save a copy of current opened file Undo revert the previous action 5 Page Setup Set the margins of page for printing including header & footer and layout of page Time/Date insert current date and time 6 Print print the current file or selected text Word Wrap Wrap text to opened notepad windows margin, Horizontal scrollbar is off 7 Exit exit from notepad Delete delete the selected text 8 Set Font format the font 9 Select All select all text in a file Table 4.1.1 Test suite of the Window 98 notepad 27
  28. 28. XP notepad Spec2# File Menu Edit Menu Format Menu View Menu 1 New to open new file Cut cut the selected text Word Wrap Wrap text to opened notepad windows margin, Horizontal scrollbar is off Status Bar Will Show status bar for this option word wrap option should be off 2 Open to open existing file size should be less then 64kb Copy copy the selected text Font format the font 3 Save Save current file Paste paste the text at cursor position 4 Save as save a copy of current opened file Undo revert the previous action 5 Page Setup Set the margins of page for printing including header & footer and layout of page Time/Date insert current date and time File Menu Edit Menu 6 Exit exit from notepad Delete delete the selected text 28
  29. 29. Set Font format the font Select All select all text in a file Table 4.1.2 Test suite of window XP notepad Comparator Comparator is basically used to compare the previous version i.e. window 98 notepad with the modified version i.e. window XP notepad. • Spec#1 represent the window 98 notepad • Spec#2 represent window XP notepad Following is basics steps of comparator: Spec#1 and Spec#2 are fed into a comparator; • The differences between the specifications based on different test coverage criteria (Spec Diff) will be generated and used to derive properties • Each specification have the two types of the properties: Hold properties (Not changed properties): Each property that holds is considered not to have changed. Not hold properties (changed properties): Each property that does not hold (a Non-Hold Prop) is considered to have changed 29
  30. 30. Figure 4.1.1 show the comparator. 30
  31. 31. 1 98 Notepad XP Notepad 2 File Menu File Menu 3 New Hold Prop New 4 open Hold Prop open 5 Save Hold Prop Save 6 Save As Hold Prop Save As 7 Page Setup Hold Prop Page Setup 8 Print Hold Prop Print 9 Exit Hold Prop Exit 10 Edit Menu Edit Menu 11 Cut Hold Prop Cut 12 Copy Hold Prop Copy 13 Paste Hold Prop Paste 14 undo Hold Prop undo 15 Time/Date Hold Prop Time/Date 16 Word Wrap non hold Replace 17 delete Hold Prop delete 18 set Font non hold Goto 19 Select All Hold Prop Select All 20 Find 21 Find Next 22 Search Format Menu 23 Find non hold Word Wrap 24 Find Next non hold Font Table 4.1.3 test suite of comparator Based on diagram 4.1.1 we have the above test suits that show the Hold and Not hold properties of 98 notepad and XP notepad. Coverage Selector Comparator discovers the differences between two versions of the specification. It works together with the coverage selector which allows user to choose test criteria that produce the Regression Test Specification (RT Spec), Coverage selector has the following specification level: a) State Transition Level b) Syntactic Trigger c) Semantic Trigger level Following test suits display the working of above three level: 31
  32. 32. Figure 4.1.2 Coverage Selector State Transition Level Syntactic Trigger Level Event Semantic Trigger Level Event diff in trigger events Events has been totally modified 1 Edit.replace Will replace the selected text with Edit.set Font Format.font Edit.wodwrap search.Find search.Find Next Format.wordWrap Edit.Find Edit.Find Next 32
  33. 33. the text in clipboard 2 New Events Added View.status Edit.Goto Table 4.1.4 Test suite of Coverage selector based on above diagram Test Generator Test Generator produces a set of counterexamples for each non-hold property. Currently, Test Generator chooses the shortest counterexample from the list, translating it into corresponding test case. Test Case ID Module/Inte rface/ Event Test Case Title Test Case Objective Test steps Expected Result 1 Edit.replace replace text Replaces the selected text with text given in find box copy the text then select the text which is going to be replaced goto Edit->Replace Selected text will be replaced by the copied text 2 Format.. Font font format the font Format menu- >font open a font dialog box with font,font style,size,script option 3 Format.wordW rap wrap Wrap text to opened notepad windows margin, Horizontal scrollbar is off Format menu- >wordwrap horizontal scroll bar will disappear 4 Edit. Find find Finds the specified text in a file Edit->find open a find dialog box 33
  34. 34. and stop at first occurrence of text asking find what text box with ok cancel button,up/down and match case options. After typing the text and selecting ok button first instance of that text will be searched 5 Edit.Find Next find next Finds the next occurrence of text entred in find box Edit->Find Next same find dialogbox will be opened to find next occurrence of that word if it reaches at the end of document and no matching word is found then alert bx will display no match found 6 View. status vstatus Will Show status bar view-> status bar Will Show status bar for this option wordwrap option should be off 7 Edit.Goto egoto Goto to specific line edit->goto goto dialog box will be opened and will ask for enter line number.For this wordwrap option should be off Table 4.1.5 Test suite of Test Generator Result RTMC are the concept of using a model checker as part of the regression test generation tool. They support a specification description, and different test coverage criteria. Most importantly, it offers a significant opportunity to reduce testing costs and automatically produce regression tests from formal specifications with greater confidence 34
  35. 35. Chapter 5 Related Work in component-based Regression testing and UML In the current scenario, focus has shifted on using UML design models to perform testing activity. We summarize existing work on generating test inputs from Regression Testing based on UML models. Work on testing using the UML has likely to be focus on generating requirements and criteria for code testing from UML diagrams. In Regression Testing UML design of software components we identify changes to UML designs to assist in classifying UML test cases. The UML designs we consider consist of UML Class Diagrams, Sequence Diagrams, and OCL expressions. The one formal implicit in our approach is that we must rely on the integrated model. In UML integrated model, The Class Diagram consists of the following primary elements: classes, attributes, methods, and associations. The Sequence Diagram contains: objects, lifelines, conditions and messages. Each of these elements can be changed by addition, deletion, or modification. Scheetz et al. [20] Towards automated support for deriving test data from UML sequence diagrams paper describes an approach to generate system test inputs from UML class diagrams. The class diagrams are restricted to contain only classes, associations and specifications and specification structures. In this approach we first identify test objectives for every single class. A test objective describes a set of objects in terms of the states they can take on. Test objectives are derived from defining desired states of the class instances after the test is executed. A state of an object is defined based on its attribute values and links to other objects. Test objectives for a complete class diagram can be aggregated from test objectives for each individual class specified in the class diagram. In this approach, we used the UML and its associated object constraint language (OCL) to describe the information about the changes from one version to next which was provided by component developer and comprised modified method and directly/indirectly affected methods, then utilized this information to direct the test case selection. This method is quite simple and can be easily implemented in practice, but this method is so rough that the selected subset of test cases generally contains many irrelevant test cases 35
  36. 36. Pilskalns et al. [15] Regression testing design paper describes the purpose of a Graph- based approach to combine the information from structural and behavioral diagrams (Class Diagram and Sequence Diagrams). In this approach, each sequence diagram is transformed into an Object-Method Directed Acyclic Graph (OMDAG). Each node in an OMDAG represents a method call or a return action, as well as the class of the object that initiates the call. The directed arcs represent control flow. OMDAGs can be used to generate the execution paths in the sequence diagram that satisfy different test adequacy criteria. UML testing technique proposed to define regression testing for UML designs. UML testing relies upon building an integrated model from Class Diagrams, Sequence Diagrams and OCL. Building the integrated model consists of three steps. The first step maps Class Diagrams into tuples, called Class Tuples (CT). The CT consists of a class name, attributes (represented as tuples if non-primitive) from the class and super classes (if applicable), and methods (represented as tuples) for the class and super classes (if applicable). A class tipple is a mathematical representation of a class and is similar to the idea of representing a class using the XMI specification. OCL statements are mapped into the CT based upon their context. The second step consists of mapping Sequence Diagrams into an Object Method Directed Acyclic Graph (OMDAG). The OMDAG maps the dynamic information in a Sequence Diagram to a directed acyclic graph. The OMDAG is created by mapping object and sequence method calls from a Sequence Diagram to vertices and arcs in a directed acyclic graph. They propose a regression testing technique based on UML sequence and class diagrams. Their approach does not take into account the pre and post conditions of the operations which affect behavior of a class. Also, their approach does not handle concurrency. So finally we have concluded that there is some lack of points in the existing approach. These points are as follow: 1. This method is quite simple and can be easily implemented in practice, but this method is so rough that the selected subset of test cases generally contains many irrelevant test cases. 36
  37. 37. 2. It’s difficult to compute the efficiency from manually UML based test case generation. 3. Above discuss approach does not take into account the pre and post conditions of the operations which affect behavior of a class. Also, their approach does not handle concurrency. In our work, we would require to extend the work of manual UML based Regression testing and will generates efficient UML regression test case generation for software component by using programmatically Graph-Based approach. 37
  38. 38. Chapter 6 Control Flow Graph (CFG) based criteria to generate Regression Test Suite. Our approach is to generate Regression Test suite for software components is based on the UML sequence diagram storage graph. Figure 6.1 shows the overall working of the control flow graph based criteria for the generation of the regression test suite. We construct the storage graph using C++ program of data structure. We have used two flow graphs; control flow graph CFG and extended current flow graph ECFG to store the UML sequence Diagram of the software components. The basic information of sequence diagram is sequence of events that are extracted from the diagram. The UML sequence diagram designed with the help of Net Beans Tool. Net Beans Unified Modeling Language (UML) tools enables you to generate UML Diagrams such as sequence diagrams. A net beans tool is the java enabled tool, there is no need to drawn the 38
  39. 39. sequence diagram manually using the net beans tool, it works as reverse engineering tool. It takes the java program source code and generates the sequence diagram automatically. The two flow graphs are compared to identify the changes and hence to generate test cases. The different steps of our approach are as follows: 1. Built Sequence diagrams for software components. 2. To construct the CFG to store sequence diagram. 3. Traversal of the Storage graph to generate regression test suites. 1. Built Sequence diagram for software components: Sequence Diagrams: A Sequence Diagrams models system behavior by specifying how object interact to complete the task. An interaction is expressed by the messages between lifelines. A lifeline is participant in an interaction. A lifeline represents a class instance. In this thesis, a message can represent a message invocation, a reply message, a creation and deletion of a class instance. Figure 6.2 shows the Basic Sequence Diagrams of Students who are enrolling in seminar in University. 2. To construct the Control Flow Graph (CFG) to store sequence diagram. In this part of our approach we construct the control flow graph( CFG) using the C++ program of data structure. The minimum requirement for the storage graph is window 98 39
  40. 40. or higher version and minimum 32 MB RAM. In the control flow graph the States are represented by vertices of graph and methods by edges through which the object changes from one state to another. The precondition and postcondition must be satisfied all the methods and their parameter are stored as edges. The construction Storage Graph is built using Graph of Data Structure, Where each Edges and Vertices are represented as State. Struct EdgeState; Struct VertexState; Struct VertexState { Char StateId [10]; int IsInitial, IsFinal; VertexState *NextVertex; EdgeState *EdgeRoot; VertexState () { IsInitial=IsFinal=0; EdgeRoot=NULL; } The composing of vertex node is as follows: State Name Next State Method Pointer Precondition Postcondtion The Edge node is as follows: Method Id State From State To Sender Receiver Attributes Next method Each attribute is abstract type and consists of: Date Type Attribute Name Algorithm for the generation of storage graph for the sequence diagram: A state is to be added in the sequence diagram. A graph in the sequence diagram and states represent the vertices of the graph. INIT is the initial state; if it is NULL then we can say that the sequence diagram is missing. 40
  41. 41. if INIT = NULL then Display “No diagram found” Endif To add a state S into graph G1 representing sequence diagram of first version, we will use it as: if INIT = NULL then INIT = S Display “first assume intail state is empty then add in it one state.” // Last state is extracted with the help of a loop traversing each state of a graph Else t = G1.lastState then t.NextState= S Endif Adding methods/ transitions into States of the control flow graph. A method represents the transition from state s1 to state s2 in graph. It has many attributes like methodid, parameters, sender, receiver, and return type of the method etc. Let s1 = source state of the transition s2 = destination state of the transition M = method of the of the each transition x = G1.findsourceState (s1) If X = NULL then Display “invalid source state” Endif End t1 = x.lasttransition ( ) // list transition in the list of transition as from state x will be extracted t1-> NextTransition = M // new method will added in the transition list. 3. Traversal of the storage graph to generate regression test suites. After written of the control flow graph we compare both version of the sequence diagram. And traverse the storage graph for both the version of the sequence diagram. We used depth first search technique for the traversal of the regression test case 41
  42. 42. algorithm. And based on the comparison of the storage graph, we generate the regression test suites. G1= Graph for first Sequence Diagram G2= Graph for Modified Version of G1 T1= Traverse Stats of G1 T2= Traverse States of G2 E1= Edges of the Graph G1 E2= Edges of the Graph G2 1. T2= G2[INIT] // initial states of Graph G2 2. while T2 != NULL // repeat for all states of graph while their exist a transition E2= T2 [METHOD POINETR] While E2 ! = NULL // repeat for all transition of current states i.e. for all methods that is applicable at current state CALL Check Match (G1, E2) // search the method E2 in G1 E2=E2 [NEXTMETHOD] // jump to next transition/method End while T2=T2 [NEXTSTATE] // jump to next state End while 3. End // ChekMatch method CheckMatch (G1, E2) // this function match the method E2 with in sequence diagram G1 1. T1= G1[INIT] Match=false // assume match is not present While (T1 != NULL) // start search match for each states and each method within state. E1= T1 [METHOD POINETR] While (E1 != NULL) 42
  43. 43. // all attributes of E1 and E2 matches If E1=E2 then Display “Reusable” Return // if method name match but attributes do not mark as true Else if E1 [MethodId] = E2 [MethodId] Match= True Endif E1= E1 [NEXT METHOD] End while // if match is true that is method was found but attributes did not matched that method is invalid If Match= True then Display “Obsolete” Else if Match= false // if nothing matches not even method name then it newly introduced method Display “test case are newly added or ReTestable” 43
  44. 44. Chapter 7 Case Studies In the previous chapter we have constructed an algorithm of control flow graph for the storage of sequence diagrams. In which, we created the states and transition for the graph. And after that, we have written an algorithm for the traversal of the storage graph for the generation of the regression test suites. Now these algorithms will be practically experimented on actual components and the component is drawn with the help of Net Beans Tool. In this chapter, we have generated the regression test cases using the sequence diagrams of the three different components. 7.1 Case Study with ATM using the Net Beans Tools for UML Sequence Diagram a) The algorithm for constructing the storage graph will store all possible information of ATM into the States and Edges, before that: First we generated the Sequence Diagram of both the version using Net Beans tools. Figure 7.1.1 shows the first version of the ATM Sequence Diagram.. And with help of the algorithm of storage graph for the sequence diagram. We store each states and transitions of the ATM sequence diagram. Source Destination Method &Parameter Sender Receiver 44
  45. 45. ST1 ST2 ST3 ST3 ST4 ST2 ST1 ST2 ST4 ST3 Request(int amount, int AccNo) getcash(int amount, string recpt) Invalid(int AccNo, String Msg) verfybal(int amount, int AccNo) balinfo(int bal, int AccNo) USER ATM CONSORT CONSORT BRANCH ATM CONSORT ATM BRANCH CONSORT Table 7.1.1 shows the case scenarios based on the ATM sequence diagram of first version. The scenarios show all the transition in which we have a different source and destinations, methods, sender and receiver. We traverse the graph and store all the information from UML Sequence diagram. 45
  46. 46. Figure 7.1.2 show extending the functionality of first version of ATM sequence Diagram. The modified version of the ATM shows some extra functionality from the previous version of the ATM. The modified component has some new events such as states and transition which was not in the previous version of the sequence diagram. Source Destination Method &Parameter Sender Receiver 46
  47. 47. ST1 ST2 ST3 ST3 ST4 ST4 ST5 ST5 ST5 ST2 ST1 ST2 ST4 ST3 ST3 ST4 ST4 ST2 Request(int amount, int AccNo) getcash(int amount, string recpt) Invalid(int AccNo, String Msg) verfybal(int amount, int AccNo) balinfo(int bal, int AccNo) clear(int AccNo, int amount) failure(int AccNo, string Msg) Success(int balance, int Recptno) Recpt(int prvbal, int currbal) USER ATM CONSORT CONSORT BRANCH BRANCH CASH_ST CASH_ST CASH_ST ATM CONSORT ATM BRANCH CONSORT CASH_ST BRANCH BRANCH ATM Table 7.1.2 shows the nine case scenarios based on the ATM sequence diagram of modified version. b) Based on the case scenarios of the both the diagrams, the algorithm of traversal will compares the events of the both the version. And generates the regression test suites. The algorithm of Regression Testing will compare both the version of the sequence diagrams or interaction diagrams. So that we can find out how many test cases are same in both the version and how many tests are new in modified version, and how many tests are not valid. TEST ID STATE METHOD (PARAMETER) SENDER RECEIVER TEST ID : 01 ST2 Request(int amount, int AccNo) USER ATM Reusable ------------------------------------------------------------------------------------------------------------------------------------ 47
  48. 48. TEST ID : 02 ST1 getcash(int amount,string recpt) ATM USER Reusable -------------------------------------------------------------------------------------------------------------------------------------- TEST ID : 03 ST3 vldacc(int AccNo,int cardno) CONSORT BRANCH Obsolete ---------------------------------------------------------------------------------------------------------------------------------------- TEST ID : 04 ST2 invalid(int AccNo,string msg) CONSORT ATM Reusable --------------------------------------------------------------------------------------------------------------------------------------- TEST ID : 05 ST4 verfybal(int 0.0000,int AccNo) CONSROT BRANCH Obsolete ----------------------------------------------------------------------------------------------------------------------------------------- TEST ID: 06 ST3 balinfo (int cbal,int AccNo) BRANCH CONSORT Reusable ------------------------------------------------------------------------------------------------------------------------------------------- TEST ID: 07 ST5 clear(int AccNo,int amount) BRANCH CASH_ST NewAdded -------------------------------------------------------------------------------------------------------------------------------------------- TEST ID: 08 ST4 failure(int AccNo,string msg) CASH_ST BRANCH NewAdded --------------------------------------------------------------------------------------------------------------------------------------------- TEST ID: 09 ST4 success(int balance,int recptno) CASH_ST BRANCH NewAdded --------------------------------------------------------------------------------------------------------------------------------------------- TEST ID: 10 ST2 recpt (int prvbal, int currbal) CASH_ST ATM NewAdded ---------------------------------------------------------------------------------------------------------------------------------------------- Table 7.1.3 shows the final Regression Test case generation from the ATM component. In regression test suites, we find the changes occur in the modified version of the ATM component. In the regression test suites half of the test cases are from the modified component. So they all are new added. 6.2 Case Study with E-Ticket of Cinema for UML Sequence Diagram Our second case study with the E-Ticket of Cinema for movie, we have generated the sequence diagram first for both the version of the component. In the E-Ticket of Cinema, the sequence diagram describes how the customer will login for ticket, and after login they can able to see the detail of the movies. They can choose the movie from the available list. 48
  49. 49. In our first step, we have generated the sequence diagram for both the version of Cinema component, Figure 7.2.1 shows the sequence diagram of first version of E-Ticket of Cinema component. Source Destination Method(parameter) Sender receiver S1 S2 Login(string Un) Customer Boooking S1 S2 TickDet(int TickNo,int seat) Customer Boooking S2 S1 LoginFail() Booking Customer S2 S3 ChFilm(string name) Booking Movie S2 S4 ChCinema() Booking Custom S2 S5 ChSeat(int seatNo) Movie SeatPlan S3 S2 AvailFilm(list film) Movie Booking S3 S4 GetMovie() Cinema Cinema 49
  50. 50. S4 S3 Details(list films) Movie Movie S5 S2 SeatAlot(list SeatNo) Seatplan Booking Table 7.2.1 shows the Case scenarios of first version of the E-Ticket system of Cinema. This case scenario is generated with the help of the storage graph algorithm; we traverse each states and transition of the graph. And in the transition we have various sender and receiver, method and their parameters. Figure 7.2.2 shows the sequence diagram of modified version of E-Ticket of Cinema. Source Destination Method(parameter) Sender receiver S1 S2 Login(string Un) Customer Boooking S1 S2 TickDet(int TickNo,int seat) Customer Boooking S2 S1 LoginFail() Booking Customer S2 S3 ChFilm(string name) Booking Movie 50
  51. 51. S2 S4 ChCinema() Booking Custom S2 S5 ChSeat(int seatNo) Movie SeatPlan S3 S2 AvailFilm(list film) Movie Booking S3 S4 GetMovie() Cinema Cinema Source Destination Method(parameter) Sender Receiver S4 S3 Details(list films) Movie Movie S5 S2 SeatAlot(list SeatNo) Seatplan Booking S5 S6 Showstat() Seatplan Seats S6 S5 Status() Seats Seatplan S1 S6 Fill_form() Customer InpForm S6 S1 Validform() InpForm Customer Table 7.2.2 shows the Case scenarios of modified version of the E-Ticket system of Cinema. Based on the comparison of the both the control flow graph of first version of the E- Ticket of Cinema and Extended flow graph of the modified version of the E-Ticket of Cinema. The traversal of the regression test algorithm will produce the final regression test suite. Test_ID State Method(Parameter) Sender Receiver Comments 01 S2 Login(string Un) Customer Booking ReUsable 02 S2 TickDet(int TickNo,int seat) Customer Booking ReUsable 03 S7 Fillform() Cinema InpForm New Added 04 S1 LoginFail() Booking Customer ReUsable 05 S3 Choose(string name) Booking Movie New Added 06 S4 ChCinema() Booking Cinema Obsolete 51
  52. 52. 07 S5 ChSeat(int seatNo) Booking Seatplan ReUsable 08 S2 AvailFilm(list film) Movie Booking ReUsable Test_ID State Method(Parameter) Sender Receiver Comments 09 S4 GetMovie Movie Cinema ReUsable 10 S3 Details (list films) Cinema Movie ReUsable 11 S2 SeatAlot(int SeatNo) Seatplan Booking Obsolete 12 S6 ShowStat() Seatplan Seats New Added 13 S5 Status() Seats Seatplan New Added 14 S1 ValidForm() InpForm Customer New Added Table 7.2.3 shows the final Regression Test case generation from the E-ticket of cinema component. 7.3 Case Study with Library Record System for UML Sequence Diagram Now practically experiment our approach with third component i.e. Library Record System, 52
  53. 53. a) Based upon the steps, first generates the sequence diagram for both of the library record system. Then possible stores those into the storage graph. And then traversal of the storage graph for regression test case selection. Figure 7.3.1 shows the Sequence diagram of the first version for the Library Record system Source Destination Method(parameter) Sender receiver S1 S2 FillForm(int form_no,int AmtFee) Person membship S2 S3 ReqIssue(int BookNo,int BName) Membship Librarian S2 S1 Verify(string PerName,string Address) Person Membship S2 S1 Register(int PerId,string msg) Membship Person S2 S3 RetBook(int BookNo,int Date) Membship Librarian S3 S2 Issued(int MembId,int BookNo) Librarian membship 53
  54. 54. S3 S2 Fine(int delay, float amount) Librarian membship S3 S4 Request(list order,float cost) Librarian PurDept Table 7.3.1 shows the case scenarios of the first version of the Library Record System Figure 7.3.2 shows the Sequence diagram of the modified version for the Library Record system. Source Destination Method(parameter) Sender receiver S1 S2 FillForm(int form_no,int AmtFee) Person membship S2 S3 ReqIssue(int BookNo,int BName) Membship Librarian S2 S1 Verify(string PerName,string Address) Person Membship S2 S1 Register(int PerId,string msg) Membship Person S2 S3 RetBook(int BookNo,int Membship Librarian 54
  55. 55. Date) S3 S2 Issued(int MembId,int BookNo) Librarian membship S3 S2 Fine(int delay, float amount) Librarian membship Source Destination Method(parameter) Sender receiver S3 S4 Request(list order,float cost) Librarian PurDept S4 S5 Order(list olderls) PurDept BookDepo S4 S5 IsAvail(list books) PurDept BookDepo S5 S4 Status() BookDepo PurDept S5 S4 Bill(float amount) BookDepo PurDept Table 7.3.2 shows the case scenarios of the modified version of the Library Record System. b) Based on the comparison of the control flow graph of first version of the library record system and extended flow graph of the modified version of the library record system. The traversal of the regression test algorithm will produce the final regression test suite. Test_ID State Method(parameter) Sender Receiver Comments 01 S2 FillForm(int formno ,int AmtFee) Person Memship ReUsable 02 S3 ReqIssue(int BookNo, int BName) Memship Librarian ReUsable 03 S1 Verify(string PerName,string Address) Memship Person Obsolete 55
  56. 56. 04 S1 Register(int PerId, string msg) Memship Person ReUsable Test_ID State Method(parameter) Sender Receiver Comments 05 S3 RetBook(int BookNo,string Date) Memship Librarian Obsolete 06 S2 Issued(int MembId, int BookNo) Librarian MembShip ReUsable 07 S2 Fine(int delay, float amount) Librarian MembShip ReUsable 08 S4 Request(list order, float cost) Librarian Purdept ReUsable 09 S5 Order(list olderls) Purdept Bookdepo New Added 10 S5 IsAvail(list books) Purdept Bookdepo New Added 11 S4 status () Bookdepo Purdept New Added 12 S4 Bill(float amount) Bookdepo Purdept New Added Table 7.3.3 shows the final regression test suite for Library record system 56
  57. 57. Chapter 8 Conclusions and Future work 8.1 Evaluation of case studies: a) Evaluation of the ATM component We can check the running time of the ATM component for producing the final regression test suite. This can be checked with the help of the bar chart. New ATM test case File Final Regression Test suite Old ATM test case File Component: ATM Total regression test cases Reusable Obsolete Retestable 10 4 2 4 10 4 2 4 0 5 10 15 Series1 10 4 2 4 Total ReUsable Obsolete Retestable Running Time: 0.011 Figure 8.1.1 shows the Bar chart of the total number of regression suite for ATM component The bar chart shows the total number of the test cases and the shows how many test cases are Retestable, Reusable, and Obsolete in regression testing along with their running time. 57
  58. 58. b) Evaluation of the E-Ticket of Cinema component We can check the running time of the E-ticket of Cinema component for producing the final regression test suite. This can be check with the help of the bar chart. . Component: E-Ticket of Cinema Total regression test cases Reusable Obsolete Retestable 14 7 2 5 New E-ticket of Cinema test file Final Regression test suite Old E-ticket of Cinema test file 14 7 2 5 0 5 10 15 Series1 14 7 2 5 Total ReUsable Obsolete Retestable Running time: 0.0264 Figure 8.1.2 shows the total number of regression suite for E-ticket of Cinema component The bar chart shows the total number of the test cases and the shows how many test cases are Retestable, Reusable, and Obsolete in regression testing along with their running time. 58
  59. 59. c) Evaluation of the Library Record System component We can check the running time of the Library Record System component for producing the final regression test suite. This can be check with the help of the bar chart. Final Regression test suite Component: Library Record System Total regression test cases Reusable Obsolete Retestable 12 6 2 4 New Library Record System test file Final Regression test suite Old Library Record System test file 12 6 2 4 0 5 10 15 Series1 12 6 2 4 Total ReUsable Obsolete Retestable Running Time: 0.0247 Figure 8.1.3 shows the total number of regression test suite for Library record system component. The bar chart shows the total number of the test cases and the shows how many test cases are Retestable, Reusable, and Obsolete in regression testing along with their running time. 59
  60. 60. 8.2 Summary of the contribution. Regression testing is defined as the execution of a set of application tests to verify that the application’s quality has not degraded. Regression testing is necessary after a change to the application program, a part of the application to a different vendor’s machine, or change in the underlying operating system. Regression testing techniques generally apply to test modified code, not designs. We have proposed a systematic technique of UML based regression testing applied on sequence diagrams. The primary goals in the design of the UML are to provide users with a ready-to-use, expressive visual modeling language so they can develop and exchange meaningful models. The UML Sequence diagrams are the most popular UML artifact for dynamic modeling, which focuses on identifying the behavior within your system. We started with the manual Regression testing for UML design. We have performed black-box as well as white box testing for the generation of the UML based regression test suite. As a part of our research work, programs have been written in C++ to generate and traverse control flow graph for the storage and traversal of UML sequence diagram. The sequence diagrams of different versions of software components are initially stored and then compared automatically to generate regression test suite. We have practically demonstrated the approach by considering three different case studies from different application domains. The results have been verified by the running time of the regression test suite generation technique. 8.3 Future Work Currently we have implemented our programs in C++ to construct control flow graph for the storage of the UML sequence diagrams and they have been traversed automatically for generation of the regression test suite. So we are only using the UML sequence diagrams for the generations of the final regression test suite. In future, to generate the regression test suite; other UML diagrams like activity and use case models can also be considered. It will widen the domain of analysis of the software component. Thus, resulting in effective test suite. 60
  61. 61. References [1] Atifah Ali1, Aamer Nadeem1, Muhammad Zohaib Z. Iqbal2, Mohammad Usman, “Regression Testing based on UML Design Models”, in the proceeding of the 10th International Conference on Information Technology, IEEE press 2007. [2] Bruce W. Weide, "Modular Regression Testing", Connections to Component-Based Software, Dept. of Computer and Information Science, The Ohio State University , 2015 eil Ave. Columbus, OH 43210,USA +1 614 292 1517 weide.1@osu.edu , IEEE press 2005 . [3] Chengying Mao, Yansheng Lu,” Regression Testing for Component-based Software Systems by Enhancing Change Information”, Proceedings of the Asia-pacific Software Engineering Conference (APSEC’05), Document number 0-7695-2465-6/05, IEEE press 2005. [4] David Binkleyf “Reducing the Cost of Regression Testing by Semantics Guided Test Case Selection”, Proceedings of the Department of Computer Science Department Loyola College in Maryland 4501 North Charles Street Baltimore, Maryland 21210 binkley @cs.loyola.edu, IEEE press 1995. [5] Demissie B. Aredo “Sematics of UML Sequence Diagrams in PVS”, in the proceeding of the Department of Informatics, University of Oslo Institute for Energy Technology. [6] Fevzi Belli, Christof J. Budnik, “Towards Self-Testing of Component-Based Software “, Proceeding of the 29th Annual International Software and Applications Conference (COMPSAC ’05), IEEE, 2005. 61
  62. 62. [7] Gregg Rothermel and Mary Jean Harrold “A Safe, Efficient Algorithm for Regression Test Selection”, Proceedings of the Department of Department of Computer Science Clemson University Clemson, SC, 29634-1906, {grot her, harrold}@cs.clemson.edu, IEEE press 1993. [8] Ghinwa Baradhi and Nashat Mansour “A Comparative Study of Five Regression Testing Algorithms”, in the procededing of the Department Ghinwa Baradhi and Nashat Mansour Computer Science Lebanese American University, nmansour@lau.edu. Lb, IEEE press 1997. [9] Hareton K. N. Leung “Insights into Regression Testing”, Proceedings of the Department of Computing Science University of Alberta Edmonton, Alberta, Canada T6G 2H1, IEEE press 2000. [10] Jiang Zheng1, Laurie Williams1, Brian Robinson2, Karen Smiley “Regression Test Selection for Black-box Dynamic Link Library Components”, Proceedings of the Department of Computer Science, North Carolina State University, Raleigh, NC 27695 {jzheng4, lawilli3}@ncsu.edu 2 ABB Inc., US Corporate Research {brian.p.robinson, karen.smiley}@us.abb.com, IEEE press 2007. [11] L. C. Briand Y. Labiche G. Soccar “Automating Impact Analysis and Regression Test Selection Based on UML Designs” , Software Quality Engineering Laboratory Carleton University Department of Systems and Computer Engineering 1125 Colonel By Drive Ottawa, ON, K1S5B6, Canada {briand} labiche}@sce.carleton.ca, IEEE press 2002. [12] Li Bao-Lin, Li Zhi-shu, Li Qing, Chen Yan Hong “Test Case automate Generation from UML Sequence diagram and OCL expression”, 15th International Conference on Advanced Computing and CommunicationsChina, IEEE 2007. 62
  63. 63. [13] Lihua Xu, Marcio Dias, and Debra Richardson “Generating Regression Tests via Model Checking” Proceedings of the Department of Informatics, School of Information and Computer Science, University of California at Irvine, CA, USA {lihuax, mdias, djr} @ ics.uci.edu, IEEE press 2004. [14] Monalisa Sarma and Rajib Mall “Automatic Test Case Generation from UML Models”, in the proceeding of the 15th International Conference on Advanced Computing and Communications , IEEE press 2007. [15] Orest Pilskalns and Gunay Uyan “Regression Testing UML Designs”, Proceedings of the Washington State University Vancouver, WA orest,gunay@vancouver.wsu.edu, IEEE press 2006. [16] Orest Pilskalns, Daniel Williams, Damir Aracic “Security Consistency in UML Designs”, in the proceeding of the School of Engineering and Computer Science Washington State University, Vancouver, WA 99686, IEEE press 2006. [17] Rami Bahsoon and Nashat Mansour “Methods and Metrics for Selective Regression Testing”, Proceedings of the Department of Computer Science, University College London, Gwer Street, London, WClE 6BT, UK. E-mail: r.bahsoon @cs.ucl.ac.uk, IEEE presses 2001. [18] Sujit Kumar Chakrabarti, and Y. N. Srikant “Specification Based Regression Testing Using Explicit State Space Enumeration”, Proceedings of the International Conference on Software Engineering Advances (ICSEA'06), IEEE press 2006 [19] Siros Supavita and Taratip Suwannasart “Testing Polymorphic Interactions in UML Sequence Diagrams”, in the proceeding of the Department of Computer Engineering, Faculty of Engineering Chulalongkorn University, Bangkok,IEEE press 2007. 63
  64. 64. [20] Scheetz et al. and j.cui “Towards Automated Support For Deriving Test Data From UML Sequence Diagrams”, in the proceeding of the 6th international conference on the UML. San Franscisco, CA, USA, October 2003. [21] Weiqun Zheng and Gary Bundell “Model-Based Software Component Testing: A UML-Based Approach”, in the procededing of the Department Weiqun Zheng and Gary Bundell Centre for Intelligent Information Processing Systems, School of Electrical, Electronic and Computer Engineering, , IEEE press 2007. 64

×