Component based testing during the software development cycle
Upcoming SlideShare
Loading in...5
×
 

Component based testing during the software development cycle

on

  • 859 views

 

Statistics

Views

Total Views
859
Views on SlideShare
859
Embed Views
0

Actions

Likes
0
Downloads
7
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Component based testing during the software development cycle Component based testing during the software development cycle Document Transcript

  • Master’s Thesis in Telematics for the Award of the Academic Degree Diplom Ingenieur at the Graz University of Technology Component based testing during the software development cycle submitted by Gerhard Fliess Institute for Information Processing and Computer Supported New Media (IICM) Graz University of Technology September 2003 Supervisor: Univ.-Doz., Dipl.-Ing., Dr.techn. Klaus Schmaranz
  • Diplomarbeit aus Telematik zur Verleihung des akademischen Grades Diplom Ingenieur an der Technischen Universit¨t Graz a Komponenten basiertes Testen im Software-Entwicklungszyklus vorgelegt von Gerhard Fliess Institut f¨r Informationsverarbeitung und Computergest¨tzte neue u u Medien (IICM) Technische Universit¨t Graz a September 2003 Begutachter: Univ.-Doz., Dipl.-Ing., Dr.techn. Klaus Schmaranz Betreuer: Univ.-Doz., Dipl.-Ing., Dr.techn. Klaus Schmaranz
  • Acknowledgments There are a lot of people that had a strong influence on what I am and who are therefore more or less responsible that I was able to finish my studies by writing this thesis. First of all I’d like to thank my parents for supporting my education and providing the freedom that is necessary to make my own decisions. My personal development over the last years was strongly influenced by the flat-sharing community I was involved in and the friends I was sailing with. Also I want to thank all friends I made music with and those who shared many hours with me while discovering Graz at night. Thanks for your time and for discussing interesting topics sometimes late at night. At the university I have to thank Klaus Schmaranz for supervising this thesis and for holding sustainable courses that had a strong influence on my view of software- development. Special thanks to Egon Valentini for joining the team that enabled CrashIt and this thesis. Additional thanks goes to Astrid for being so patient during the final editorial steps of this work. Gerhard Fliess Graz, October 3, 2003 iii
  • iv I hereby certify that the work reported in this thesis is my own and that work performed by others is appropriately cited. Signature of the author: Ich versichere hiermit wahrheitsgem¨ß, die Arbeit bis auf die dem Aufgabensteller a bereits bekannte Hilfe selbst¨ndig angefertigt, alle benutzten Hilfsmittel vollst¨ndig a a und genau angegeben und alles kenntlich gemacht zu haben, was aus Arbeiten an- derer unver¨ndert oder mit Ab¨nderungen entnommen wurde. a a
  • Abstract The IT community has learned how to design and implement highly reusable classes during the past ten years. This was forced by writing down the different design patterns and defining the component based approach. Now another aspect becomes more and more important: do these classes behave like it has been defined during the design-phase? Existing test-benches are based on the approach that test-classes must be imple- mented or additional test code in the implementation has to be written. This code will include the same amount of errors than the implementation code so another type of defining test cases can be usefull to avoid this errors. Defining the tests in a descriptive way e.g. by writing down an XML file has the advantage that developers can specify the test in a languageindependent format that simplifies compatibility tests. These files describe what should be done during the test without implementing the neseccary steps. The files can be created before all features of the component that should be tested have been implemented. This thesis covers some theoretical topics on testing and software design and intro- duces a framework that is designed to fulfill the requirements of a modern software test-bench. v
  • Kurzfassung Die IT-Gemeinde hat in den letzten 10 Jahren gelernt wie man vielf¨ltig wiederver- a wendbare Klassen entwirft und implementiert. Das wurde durch die Definition der Design Patterns und das Aufkommen des komponentenbasierten Ansatzes verst¨rkt. a Nun wird ein weiterer Aspekt immer wichtiger, n¨mlich, ob sich die so entwickelten a Klassen wirklich so verhalten, wie es das Design vorgibt. Bestehende Testumgebungen verfolgen den Ansatz, dass spezielle Testklassen be- nutzt werden oder zus¨tzlicher Programmcode f¨r die Tests implementiert wer- a u den muss. Diese zus¨tzlichen Programmzeilen haben dieselbe Fehlerrate, wie der a eigentliche Programmcode. Darum muss ein anderer, weniger fehleranf¨lliger Weg a gefunden werden, Tests zu definieren. Die Definition der Testf¨lle in einer beschreiben- a den Art, z.B. als XML-Datei, hat den Vorteil, dass die Testf¨lle unabh¨ngig von der a a Sprache beschrieben werden. Dieser Ansatz erleichtert Kompatibilit¨tstests. Die a Testbeschreibungen k¨nnen dar¨ber hinaus schon erstellt werden, wenn noch nicht o u alle zu testenden Klassen implementiert sind. Sie beschreiben, was w¨hrend des a Tests zu tun ist, ohne die einzelnen Vorg¨nge zu implementieren. a Diese Arbeit behandelt einige theoretische Aspekte der Softwareentwicklung und des Testens und stellt ein Framework vor, das erstellt wurde, um die oben genannten Anforderungen an ein modernes Software-Testwerkzeug zu erf¨llen. u vi
  • Table of Contents Acknowledgments iii Abstract v Kurzfassung vi 1 Introduction 1 2 Motivation 4 2.1 Starting Point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Approach to a Solution . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.3.1 jUnit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.3.2 iContract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3 Aspects of calls and contracts 14 3.1 Method-calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.1.1 Types of calls . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.1.2 Components and States . . . . . . . . . . . . . . . . . . . . . 15 vii
  • TABLE OF CONTENTS viii 3.1.3 Method-call model . . . . . . . . . . . . . . . . . . . . . . . . 16 3.1.4 Sequence of calls . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.2 Contracts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.3 State-machines as sequence-models . . . . . . . . . . . . . . . . . . . 20 3.3.1 Limitations of state-machines . . . . . . . . . . . . . . . . . . 20 3.3.2 Enhanced state-machines . . . . . . . . . . . . . . . . . . . . . 22 3.4 The categorization of contracts . . . . . . . . . . . . . . . . . . . . . 22 3.4.1 Linear Contracts . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.4.2 Pseudo Linear Contracts . . . . . . . . . . . . . . . . . . . . . 23 3.4.3 Looped Contracts . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.4.4 States and pre-conditions . . . . . . . . . . . . . . . . . . . . . 25 3.4.5 How to combine components . . . . . . . . . . . . . . . . . . . 25 4 General software quality factors 27 4.1 A small example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.1.1 Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.1.2 Implementations and usage of the framework . . . . . . . . . . 32 4.2 Important Quality Factors . . . . . . . . . . . . . . . . . . . . . . . . 33 4.2.1 External factors . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.2.2 Internal Factors . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.3 Design patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4.3.1 Hook and HookUp . . . . . . . . . . . . . . . . . . . . . . . . 40 4.3.2 Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 4.4 Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
  • TABLE OF CONTENTS ix 5 Testing 45 5.1 Goals for running tests . . . . . . . . . . . . . . . . . . . . . . . . . . 46 5.1.1 Targets of test runs . . . . . . . . . . . . . . . . . . . . . . . . 47 5.2 Test-design concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 5.2.1 Test principles . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 5.2.2 Descriptive or declarative configurations . . . . . . . . . . . . 53 5.3 Exception safety and robustness . . . . . . . . . . . . . . . . . . . . . 54 6 CrashIt - A short introduction 56 6.1 What is CrashIt ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 6.2 Main concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 6.3 Testing topics covered by CrashIt . . . . . . . . . . . . . . . . . . . . 59 6.3.1 Correctness . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 6.3.2 Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 6.3.3 Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 7 Using CrashIt in the software design cycle 62 7.1 Methodology concepts . . . . . . . . . . . . . . . . . . . . . . . . . . 63 7.2 Development-models . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 7.2.1 The waterfall model . . . . . . . . . . . . . . . . . . . . . . . 64 7.2.2 The spiral model . . . . . . . . . . . . . . . . . . . . . . . . . 66 7.2.3 Prototyping . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 7.2.4 Extreme programming . . . . . . . . . . . . . . . . . . . . . . 68 7.3 Using CrashIt in different models . . . . . . . . . . . . . . . . . . . . . 71
  • TABLE OF CONTENTS x 7.3.1 Which way is the best? . . . . . . . . . . . . . . . . . . . . . . 71 8 The Implementation of CrashIt 73 8.1 Overall architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 8.2 Interfaces and design-decisions . . . . . . . . . . . . . . . . . . . . . . 75 8.2.1 The CrashIt environment . . . . . . . . . . . . . . . . . . . . . 75 8.2.2 Accessing the configuration, the configuration-layer . . . . . . 77 8.2.3 Using the framework . . . . . . . . . . . . . . . . . . . . . . . 78 8.2.4 Interfaces for test-cases . . . . . . . . . . . . . . . . . . . . . . 80 8.2.5 The XML-subsystem . . . . . . . . . . . . . . . . . . . . . . . 81 8.3 Available applications . . . . . . . . . . . . . . . . . . . . . . . . . . 83 8.3.1 Stand-alone Application . . . . . . . . . . . . . . . . . . . . . 84 8.3.2 Ant-task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 9 Conclusion 85 9.1 Personal experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 9.2 Project related topics . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 9.3 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 Appendix 88 A The usage of CrashIt - a simple example 88 A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 A.2 A service example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 A.2.1 Parts of the framework . . . . . . . . . . . . . . . . . . . . . . 89
  • TABLE OF CONTENTS xi A.2.2 Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 A.2.3 Implementations . . . . . . . . . . . . . . . . . . . . . . . . . 94 A.3 The CrashIt Configuration . . . . . . . . . . . . . . . . . . . . . . . . 96 A.3.1 The Configuration file . . . . . . . . . . . . . . . . . . . . . . 97 A.3.2 The Test-configuration File . . . . . . . . . . . . . . . . . . . 100 A.3.3 The Testcase files . . . . . . . . . . . . . . . . . . . . . . . . . 104 A.3.4 The Component Files . . . . . . . . . . . . . . . . . . . . . . . 106 A.3.5 The Connection Files . . . . . . . . . . . . . . . . . . . . . . . 107 A.3.6 The Contract Files . . . . . . . . . . . . . . . . . . . . . . . . 111 A.3.7 Creating Objects - the Parameter-Converter . . . . . . . . . . 111 A.3.8 Flowcontrol . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . III Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VIII
  • List of Figures 3.1 Model of a deterministic call . . . . . . . . . . . . . . . . . . . . . . . 17 3.2 Model of a nondeterministic call . . . . . . . . . . . . . . . . . . . . . 18 3.3 A one-way state-machine . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.4 State-machine for permutations of calls . . . . . . . . . . . . . . . . . 21 3.5 State-machine similar to a one-way state-machine . . . . . . . . . . . 21 3.6 An enhanced state-machine . . . . . . . . . . . . . . . . . . . . . . . 22 3.7 Linear contract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.8 A pseudo linear sequence . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.9 A looped sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.10 A call of a deterministic method . . . . . . . . . . . . . . . . . . . . . 26 3.11 A call of a nondeterministic method . . . . . . . . . . . . . . . . . . . 26 7.1 Elements of a methodology . . . . . . . . . . . . . . . . . . . . . . . . 63 7.2 The waterfall model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 7.3 The spiral model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 7.4 Life-cycle in XP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 8.1 CrashIt modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 xiii
  • LIST OF FIGURES xiv 8.2 UML diagram of the environment classes . . . . . . . . . . . . . . . . 77 8.3 UML diagram of internal configuration-interfaces . . . . . . . . . . . 79 8.4 UML diagram of the configuration tags (incomplete) . . . . . . . . . 83
  • Listings 2.1 jUnit vector test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2 Implementation of a stack enhanced by iContract . . . . . . . . . . . 11 8.1 CrashIt tag-library example . . . . . . . . . . . . . . . . . . . . . . . . 82 8.2 CrashIt as ant-task . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 A.1 Service interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 A.2 ServiceProvider Interface . . . . . . . . . . . . . . . . . . . . . . . . . 92 A.3 Configurator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 A.4 A sample application without a Configurator . . . . . . . . . . . . . 95 A.5 A sample application using a Configurator . . . . . . . . . . . . . . . 96 A.6 The main configuration file . . . . . . . . . . . . . . . . . . . . . . . . 98 A.7 A simple configuration without Flowcontrol . . . . . . . . . . . . . . 101 A.8 A Flowcontrolled configuration . . . . . . . . . . . . . . . . . . . . . . 101 A.9 A simple Testcase-sequence without Flowcontrol . . . . . . . . . . . . 103 A.10 A Testcase-sequence using Flowcontrol . . . . . . . . . . . . . . . . . 103 A.11 Calling a method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 A.12 Description of a component . . . . . . . . . . . . . . . . . . . . . . . 107 A.13 The connections config file . . . . . . . . . . . . . . . . . . . . . . . . 108 xv
  • LISTINGS xvi A.14 Connections file with direct connections . . . . . . . . . . . . . . . . . 108 A.15 A connection that uses a Contract . . . . . . . . . . . . . . . . . . . . 109 A.16 Definition of the service-Contract . . . . . . . . . . . . . . . . . . . . 111 A.17 Definition of the used Contracts . . . . . . . . . . . . . . . . . . . . . 111 A.18 Converter property file . . . . . . . . . . . . . . . . . . . . . . . . . . 112
  • Chapter 1 Introduction What would life be if we had no courage to attempt anything? Vincent van Gogh If somone is about to solve a problem by writing an application, it is important to write down the problem and work out the main aspects. After the problem has been specified in that way, it is useful to verify wether someone else has solved it before. Sometimes good solutions are found which can be easily adapted to solve the problem but not always do the available solutions cover all important aspects. This is the time to start your own implementation. I stumbled over the problem of testing applications that are implemented by a component-based approach during a lecture held byby Klaus Schmaranz at the Graz University of Technology. During a seminar-project two other colleques and I tried to solve it, but we had to stop at the point of definig the main aspects of this problem. We accepted as a result that no solution covers all our requirements or is exentsible enough to adapt it in a way that it can fulfill our requirements. One exciting year later the first prototype of a new test-bench was released. Egon Valentini and I had implemented it as a core-framework which fulfills most of our 1
  • CHAPTER 1. INTRODUCTION 2 requirements and which is extensible enough to form the basis of more complex test-benches. It provides a more general approach of defining test-cases than other frameworks and includes interfaces for implementing custom configurations which can thus be integrated into larger software development applications. Many people associate “testing” with proving that an application runs correctly. In- deed, testing is a destructive process with the aim of findinig bugs and thus crashing an application in a secure testing environment. Using this definition as a starting point, the created test tool is supposed to motivate programmers to detect more bugs and finally to create better programs. According to these ideas it was named CrashIt . Integrating the test into the design-process is one key-factor for a succesful test- process and therefore essential for a reliable and stable product. CrashIt simplifies the integration of the test-design into early phases of the development-cycle. There several approaches for solving the requirements of such test applications. However, no test-application which could accompany engineers through the whole design and implementation cycle of a software project exists. On suggestion of Klaus Schmaranz, Egon Valentini and I hence created a tool, that helps software developers to verify their designs and to test their implementations through an ex- tensive and complete check. It includes both unit- and integration-tests in a single session. Furthermore, we realized that the idea behind this test tool reveals a po- tential to change the status of testing in the IT community. The design is based on the idea to enhance the status of testing from a neglected to a widely recognised one. CrashIt was designed and implemented in teamwork by Egon Valentini and me (Ger- hard Fliess) between May 2002 and June 2003. The main modules of the framework were developed together. This and Egon’s theses ([Val03]) are the results of our
  • CHAPTER 1. INTRODUCTION 3 teamwork. Egon describes in his thesis the design of the framework, the module interconnections (refer to [Val03, chapter 6, p.62]) and the implementation of the main modules and Flowcontrol (refer to [Val03, section 5.1.3, p.49]). Futhermore it contains a general introduction into CrashIt , a critical evaluation of the project and takes a look on the future of this application. In addition, he explains the main modules and some use-cases [Val03, chapter 6, p.62]). I was responsible for the runtimeenvironment (also called CrashItEnvironment (see 8.2.1), a method to optimally describe tests through XML-Files and I implemented the modules that are used by CrashIt to generate test reports, i.e. I wrote all parsers and document generation classes. In my thesis I will discuss the usage of CrashIt in different software-design models as well as the principles of testing and explain the runtime-environment of this test tool. As these two theses cover the same software project both cover some equal topics. We shared some topics and wrote some general sections only once. This applies to chapters 1,4,5 and 6. Chapter 1 was written to equal parts by Egon and me, chapter 4 was mainly written by Egon except for the sections on design patterns and exceptions. Chapter 5 was written by me and chapter 6 includes only few parts of the complete chapter which can be found in Egon’s thesis. Klaus Schmaranz supervised our theses and we agreed on this proceeding after releasing the first version of CrashIt .
  • CHAPTER 2. MOTIVATION 4 Chapter 2 Motivation People forget how fast you did a job- but they remember how well you did it. Howard W. Newton 2.1 Starting Point During the development of an application some phases are more interesting and challenging than others. Most developers preferred designing and implementing an application than writing the documentations or installation manuals. The most condemned task is testing the application. The starting point which is to invite to have a critical look at testing, is excellently described by Kent Beck: Testing Strategy: Oh yuck. Nobody wants to talk about testing. Testing is the ugly stepchild of software development. The problem is, everybody knows that testing is important. Everybody knows they don’t do enough testing. [...] Remember the principle “Work with human nature, not against it.” That
  • CHAPTER 2. MOTIVATION 5 is the fundamental mistake in the testing book I’ve read. They start with the premise that testing is at the center of development. You must do this test and that test and oh yes this other one, too. If we want programmers and customers to write tests, we had better make the process as painless as possible, realizing that the tests are there as instrumentation, and it is the behavior of the system being instrument that everyone cares about, not the tests themselves. [...] Tests are most valuable when the stress level rises, when people are working too much, when human judgment starts to fail. So the tests must be automatic - returning an unqualified thumbs up/thumbs down indication of whether the system is behaving. [Bec00, Chap 18] In other words, Kent Beck explains that software engineering needs tools • to test parts of an application and whole applications automatically • resulting in the generation of a test report. Moreover, there is also a need for • a tool which developers may use during the whole life-cycle of a software-pro- ject, • computer-aided generation of test-cases from a specification and [Bec00] • testing using contracts [Mey97, Chap. 12-14]. 2.2 Approach to a Solution In order to provide a solution to this situation CrashIt was created. It consists of a Java-framework, which can automatically test Java-components on the basis of test-cases. After all test had be run it generates a test-report.
  • CHAPTER 2. MOTIVATION 6 In CrashIt an automatic test doesn’t cover the generation of test-cases. Automatic test means in this case that it runs the a test procedure composed of: 1. testing of single components 2. loading, connecting according components via Contracts and testing the root- component. (Contracts are used to check, whether conditions between two components hold during communication) 3. loading, combining components to a complete application and trying to find bugs, too. The user has to specify the test-cases before running a test. The goal of writing test- cases manually is that the test-engineer is able to get a feeling which tests are useful and which test are futile ones. For example, if a method computes the addition of two values, it would be useless to test this method for each permutation of these two values. Whereas appropriate tests consist in inspecting the method and trying to find those permutations where the addition fails, e.g. value1 = maxInteger and value2 = maxInteger, so if the method-result returned by an integer value, the addition of value1 and value2 produces an overflow. The goal of designing appropriate test-cases is understanding and scrutinizing a sys- tem in terms of correctness, robustness, extendibility, compatibility and performance (see also section 4.2 on page 33). CrashIt meets those premises by an extended pos- sibility how test-cases can be to formulated. It also provides a mechanism to group test-cases into simple or complex sequences, so called Testcase-sequences. Not only test-cases but also sequences of test-cases can crash a system. Moreover, CrashIt has the capability to organize unit- and integration-test in one test. Well designed systems can be split up into several parts, socalled modules that possibly consist of one ore more components. To perform a detailed test, each com- ponent and all possible combinations of them have to be checked. CrashIt can test all components at once and further run tests in connected components. The con- nections between the components must be specified in the configuration and CrashIt
  • CHAPTER 2. MOTIVATION 7 will create the connections before the test is started. A new approach to create modular software which is moreover correct and robust, consists in defining contracts between according components. This approach is also known under the synonym Design by Contract. The advantage of Design by Con- tract is that most components shrink, because through predefined conditions it can be taken for granted that some cases do not appear when the component will be processed. Supplementary checks can consequently be dropped. As a consequence, when the program decreases, it is easier keep an overview of its components and to maintain the code. (see also [Val03, chapter 4, p.34] and [Mey97]). During a test CrashIt supports Design by Contract through the possibility to insert Contracts between components. These Contracts are simple Java classes which had to fulfill some simple precepts that are caused by the fact, that Java do not support multiple inheritance from classes. Contracts have to: • implement the Contract interface • and to implement either the interfaces of both components • or they have to be derived from one component and implement the interface of the second one CrashIt is able to load contracts dynamically. It is possible to deliver them for addi- tional tests but in general they are not part of a packaged application. Finally, a very important requirement is the ability to compare test results with expected results. This is an easy task for simple result-types like integer. Whereas it fast raises to a complex job, if user-types have to be validated. The approach implemented in CrashIt consists of special Result-checker-classes. They are Java classes, that can be extended for own purposes. Result-checker-classes for simple types are already implemented.
  • CHAPTER 2. MOTIVATION 8 2.3 Related Work As the topic of writing stable and reliable software is getting more and more impor- tant several approaches for testing and designing such applications are available. In some software-development models the importance of testing has been increased as it is an important part of the process, e.g. eXtreme Programming forces developers to implement tests as the first step in a development-cycle. CrashIt is therefore not the first approach to support developers in writing stable and reliable applications. jUnit and iContract are two interesting projects that deal with these topics in different ways. 2.3.1 jUnit jUnit is a small but powerful framework for writing repeatable tests that has been designed by Erich Gamma and Kent Beck [jUn03]. It consists of two main classes: the TestCase and the TestSuite. Both implement the same interface Test that allows to start a test by invoking the method run. A TestCase performs one or more tests by invoking method calls and comparing the results of these calls with expected values. Concrete TestCases can be implemented by extending the class TestCase and by implementing methods whose names start with test. If a concrete TestCase class is applied to the framework, jUnit searches for those special named methods and invokes them. TestCases can be successful, cause an error or cause a failure in which case the test cannot be started. The result of a test run can be formatted by a TestFormatter and used as a report of the applied tests. A TestSuite can combine and start several TestCases. The method that is used to add a TestCase requires an instance of type Test and therefore a TestSuite can also combine several TestSuites. This simple design allows a flexible structure of test-classes. There are also methods for setting up and shutting down tests, so that it is possible to initialize needed environments for them.
  • CHAPTER 2. MOTIVATION 9 1 package j u n i t . s a m p l e s ; 2 3 import j u n i t . framework . ∗ ; 4 import j a v a . u t i l . V e c t o r ; 5 6 public c l a s s V e c t o r T e s t e x t e n d s TestCase { 7 protected V e c t o r fEmpty ; 8 protected V e c t o r f F u l l ; 9 10 public s t a t i c void main ( S t r i n g [ ] a r g s ) { 11 j u n i t . t e x t u i . TestRunner . run ( s u i t e ( ) ) ; 12 } 13 protected void setUp ( ) { 14 fEmpty= new V e c t o r ( ) ; 15 f F u l l = new V e c t o r ( ) ; 16 f F u l l . addElement (new I n t e g e r ( 1 ) ) ; 17 f F u l l . addElement (new I n t e g e r ( 2 ) ) ; 18 f F u l l . addElement (new I n t e g e r ( 3 ) ) ; 19 } 20 public s t a t i c Test s u i t e ( ) { 21 return new T e s t S u i t e ( V e c t o r T e s t . c l a s s ) ; 22 } 23 public void t e s t C a p a c i t y ( ) { 24 int s i z e = f F u l l . s i z e ( ) ; 25 f o r ( i n t i = 0 ; i < 1 0 0 ; i ++) 26 f F u l l . addElement (new I n t e g e r ( i ) ) ; 27 a s s e r t T r u e ( f F u l l . s i z e () == 100+ s i z e ) ; 28 } 29 30 public void t e s t C o n t a i n s ( ) { 31 a s s e r t T r u e ( f F u l l . c o n t a i n s (new I n t e g e r ( 1 ) ) ) ; 32 a s s e r t T r u e ( ! fEmpty . c o n t a i n s (new I n t e g e r ( 1 ) ) ) ; 33 } 34 public void t e s t E l e m e n t A t ( ) { 35 I n t e g e r i = ( I n t e g e r ) f F u l l . elementAt ( 0 ) ; 36 assertTrue ( i . intValue () == 1); 37 38 try { 39 f F u l l . elementAt ( f F u l l . s i z e ( ) ) ; 40 } catch ( ArrayIndexOutOfBoundsException e ) { 41 return ; 42 } 43 f a i l ( ” Should r a i s e an ArrayIndexOutOfBoundsException ” ) ; 44 } 45 public void t e s t R e m o v e A l l ( ) { 46 f F u l l . removeAllElements ( ) ; 47 fEmpty . r e m o v e A l l E l e m e n t s ( ) ; 48 a s s e r t T r u e ( f F u l l . isEmpty ( ) ) ; 49 a s s e r t T r u e ( fEmpty . isEmpty ( ) ) ; 50 } 51 52 } Listing 2.1: jUnit vector test This approach is simple to use and can be learned easily. Some development tools integrate wizards for defining and applying jUnit tests on applications. jUnit is widely used and there are different extensions for testing web- or J2EE (Java 2 Enterprise Edition) - applications. A small example which is part of the sample package of jUnit is shown in Listing 2.1 on page 9.
  • CHAPTER 2. MOTIVATION 10 Benefits and Weaknesses We claim that the sourcecode of test-classes contains a nearly equal amount of fail- ures, test-classes written in this way are as buggy as the original code. In a typical project at least as much test code as program source code will be written and there- fore this approach might be dangerous. It is true that the test-classes are often used, but how often will they be reviewed? A solution for this problem can be to define the tests in another way than program- ming test classes, e.g. in XML files. A further aspect is that it is more secure to specify what should be done in a test than to implement it. This approach may cause some limitations for the test design, but if the configuration possibilities are flexible enough, this should not be a real restriction. The definition of a test in a separated, non-Java source file also has the advantage that it can be created automatically or it can be the source for an automatically generated test report. jUnit is a well established framework with a huge amount of related projects. An- other advantage of jUnit is that it can be integrated easily into an IDE (Integrated Development Environment) as it has already been done for Eclipse1 . 2.3.2 iContract iContract is a Java tool that provides developers with support for Design by Con- tract(DbC). DbC refers to the concept of considering the interfaces between system components as contracts that are specified as integral parts of the sourcecode. Until today, the explicit specification of contracts by means of class invariants and mes- sage pre- and postconditions was available for Eiffel and some formal specification languages like VDM2 . iContract is a prototype tool that provides similar support for Java. It enables developers to take advantage of the following benefits: 1 Eclipse, see online http://www.eclipse.org 2 see online http://www.ifad.dk/Products/vdmtools.htm
  • CHAPTER 2. MOTIVATION 11 1. support of design for testability by enhancing the system’s observability (fail- ure occurs close to fault), 2. uniform implementation of invariant-, pre- and postcondition checks among members of your team, 3. documentation and code are always in sync and 4. semantic level specification of what requirements/benefits a class/message of- fers. iContract is a freely available sourcecode preprocessor, which instruments the code with checks for class invariants, pre- and postconditions that may be associated with methods in classes and interfaces. Special comment tags (e.g. @pre, @post) are in- terpreted by iContract and converted into assertion check code that is inserted into the sourcecode. The semantic of iContract includes also quantifiers (forall, exists) to specify properties of enumerations, implications, old- and return-value references in postconditions, as well as the naming of exception classes to throw. iContract supports the propagation of invariants, pre- and postconditions via inheritance and multiple interface implementation, as well as multiple interface extension mecha- nisms. Due to the non-mandatory nature of the comment tags, source code that contains DbC annotations remains fully compatible with Java and can be processed with standard Java compilers enabling a risk-free adoption of the technique. [Kra01] A small example: 1 /* * 2 @inv ( top >= 0 && top < max ) 3 */ 4 c l a s s MyStack 5 { 6 private O b j e c t [ ] e l e m s ; 7 private i n t top , max ; 8 9 /* * 10 @pre ( sz > 0) 11 @post ( max == sz && elems != null ) 12 */ 13 public MyStack ( i n t s z ) 14 { 15 max = s z ; 16 e l e m s = new O b j e c t [ s z ] ; 17 }
  • CHAPTER 2. MOTIVATION 12 18 19 /* * 20 @pre ! isFull () 21 @post ( top == $prev ( top ) + 1) && elems [ top -1] == obj 22 */ 23 public void push ( O b j e c t o b j ) 24 { 25 e l e m s [ top ++] = o b j ; 26 } 27 28 /* * 29 @pre ! isEmpty () 30 @post ( top == $prev ( top ) - 1) && $ret == elems [ top ] 31 */ 32 public O b j e c t pop ( ) 33 { 34 return e l e m s [−−top ] ; 35 } 36 37 /* * 38 @post ( $ret == ( top == max )) 39 */ 40 public b o o l e a n i s F u l l ( ) 41 { 42 return top == max ; 43 } 44 45 /* * 46 @post ( $ret == ( top == 0)) 47 */ 48 public b o o l e a n isEmpty ( ) 49 { 50 return top == 0; 51 } 52 } // End MyStack Listing 2.2: Implementation of a stack enhanced by iContract Benefits and Weaknesses iContract compiles pre-, postconditions and invariants into assertion-checks, that can be added to the original program. This additional code causes an application to slow down a little bit so that time-critical sections may crash. It could be a complex task whether conditions or invariants have to be specified for user types or a check implicates complicate calculations. The syntax of iContract is not as powerful as Java or Eiffel to permit unlimited possibilities to express condi- tions. A bad workaround would consist of specifying Java methods that implement the check. Furthermore these methods would be used by some statements in iCon- tract to express conditions. A benefit of iContract is the small implementation effort of conditions, because they can be simply expressed in some comment tags.
  • CHAPTER 2. MOTIVATION 13 As mentioned in the past sections both projects have its benefits and weaknesses. Both are interesting projects that are useful in many cases. However, they are not able to address all requirements that lead to the implementation of CrashIt . CrashIt is designed to combine unit- and integration- tests with the concepts of con- tracts. This was done in a way that allows a smooth integration of this concepts into existing development processes. One topic of this thesis is how CrashIt can be used and how it fits into different develoment-models This will be explained in later chapters and an extensive example of its usage can be found in the appendix (see page 88). The next sections will address some theoretical concepts in order to change developers view on software- design.
  • CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 14 Chapter 3 Aspects of calls and contracts If you be in a hurry, take the longer way. Lao Tse Sometimes it makes sense to lean back and think of concepts that are used over and over again to get a new view on them. Even reinventing things often opens a new angle and helps to gain a deeper understanding of the related topics. This chapter tries to open a new view on method-calls and contracts to facilitate developers in making their design decisions. 3.1 Method-calls Calling a method of a component is a very basic process, but it is not as simple as it seems. Each method must be called by using the correct parameters and will often return a value or an object reference. There are a lot of possible reasons why a call may cause an error. Some of these reasons, like an incorrect signature, can be checked by the compiler. Other reasons cannot be checked by the compiler because it is not possible to specify them in the
  • CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 15 chosen language. For example, there can be dependencies in the used parameters as there are two integer parameters and the first parameter must be greater than the second one. Other reasons for an error during a call can be a busy device, an incorrect initialized object, a reference to a deleted object etc. The only thing a developer can do is specify the behavior of the object, when the call cannot be finished successfully. The behavior is defined by an exception, so that it is possible to react to this error. Some methods are very simple or do not use some any critical resources, so that it is not possible for an error to occur during a call. 3.1.1 Types of calls According to this view there are two types of method-calls. Calls that will deter- mine successfully in either case and method-calls that can be interrupted because something is going wrong. To make it easier to distinguish between these types, they will be named deterministic call and nondeterministic call . Developers often concentrate on the deterministic calls and forget to define the be- havior of the nondeterministic ones. Both method types are equally important for the design process and also for testing the software. It is important to accept an exception as a valid result of a call. The difference between an exception and a result of a successful call is that the execution of the program will follow a different way. An invalid call would be if an error is not reported by an exception. This often causes a crash of the whole application. 3.1.2 Components and States One basic assumption is that each component has internal states. Developers often do not recognize these states and, consequently, there is no representation of these states in the program. Each component has at least two states, justCreated and deleted. Depending on the functionality there may be only these two states or a lot of other states between them. Sometimes flags or status variables are used to
  • CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 16 store state information. Even when they are implemented, they cannot be used for testing, as there can be a mistake in the coide that isz used to set the state infromation and everything seems to be right although an error has occurred. That is why the only reliable way of testing is to monitor the behavior of the class. Before describing the behavior of a component by defining a sequence of calls we should take a closer look at method-calls and define a model for them. 3.1.3 Method-call model Calling a method changes the internal state of a component. This can result in a new state or the component can stay in the same one. During the call the compo- nent changes into an intermediate state in which the component calculates the result of the method call. Sometimes it is necessary to consider this state, but often the component stays in this state for a very short time and therefore it can be ignored. When a method-call is blocking this state must be considered. A further example for considering this intermediate state is, when the method is solving a concurrency problem. In this case the method has to be defined assynchronized, which means that only one call of this method can be done as long as the execution of the method has not been finished. Another important point is that during the execution of a method an error can oc- cur. This will probably cause the component to switch into a state that cannot be directly reached from the original state. This seems to be a needlessly complicated view of method calls, but this model allows a simpler test-framework and it is im- portant to understand this view of method-calls to work with the test-framework. The test-framework becomes simpler because this model defines deterministic state- machines for this nondeterministic problem of calling methods. Here are some rules for understanding the following graphics and the more enhanced ones in later chapters.
  • CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 17 • circle: Equals a state. • line with arrow: Equals a transition between two states. The arrow shows the direction and points at the result state • solid line: Equals a normal transition. This transition is the reason why the method was written. • dotted items: Denote internal intermediate states or transitions that are used by the model. Transitions caused by events like exceptions must be specified for error handling in order to make it possible to test the correct behavior. • rectangle: Equals an event, like a method-call or the occurrence of an excep- tion. In section 3.4.5 we will see that an event can have more than one effect, however, up to now the only effect is that the transition is executed. Model of deterministic calls In this example the component is in the state A before a method is called. During the call the component is in the intermediate state A*. After the result has been calculated there is only one possible state, in this example labeled as B. Figure 3.1: Model of a deterministic call Model of nondeterministic calls A nondeterministic call is quite equal to a deterministic call except for the fact that the intermediate state A* has two possible transitions into further states. If the
  • CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 18 result can be calculated, the component will switch into state B. If an error occurs, the component will switch into the state C. Figure 3.2: Model of a nondeterministic call 3.1.4 Sequence of calls It is very important to describe the usage and the behavior of components both in order to use and to test the component. Thinking of the description of a component and considering the testability during the design process makes it easier to design simpler and therefore better components. If it is easy to test a component, it is probably easy to use it. Otherwise components that do not facilitate there tests often include design problems. The behavior and the allowed usage can be described by a sequence of method-calls and their results. Some methods cannot be called, when a certain state has not been reached. These calls define a possible sequence of method-calls which is the only way the component can be used. Sequences of method calls are the first step to define contracts of components. 3.2 Contracts When a component is implemented, there is no direct way to specify its usage and behavior. The usage is written down in the documentation of a component.
  • CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 19 Documentations are human readable, but it is difficult to write a program that extracts test-cases for the component from the documentation. A good way of solving this problem is to define contracts between components. These contracts can be written in a machine readable form and therefore it s possible to transform them into human readable texts or diagrams. One part of a contract is the way of calling methods of a component. Depending on the language exceptions may be thrown when an error occurs. Contracts can not solve the problem that has caused the exception, but they can specify a way of dealing with it. A correct signature cannot prohibit an illegal call of a method. Methods often need some pre-conditions which must be fulfilled before a method can be used. After a successful call the result has to match a post-condition and the state of the component may have changed. Contracts are defined by signatures and a possible sequence of method- calls Describing the correct sequence can be difficult, as there may be many variants. A good design should prevent complex sequences and limit the amount of possible variants. One good approach to describe the correct sequences is to define the start- and the end-state for each method-call. In some cases it is useful to consider the intermediate states. The description can be defined as set of states combined with a set of methods and transitions between them. • S set of states • Ss ⊂ S set of start states • Se ⊂ S set of end states • M set of methods • Ss × M → S e
  • CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 20 In this model transitions correspond to method calls. There are various types of contracts which can be classified by taking a look at the time line of permissible method-calls. This will be modeled by using a state-machine. 3.3 State-machines as sequence-models State-machines are well-known constructs to control different processes. They are defined by states and transitions between these states. [ASU99, p. 137]1 A state- machine does not remember the earlier states, it only knows the actual state. This behavior is not enough for modeling the allowed sequences of method calls, and there are other limitations, too, so that it is necessary to enhance the functionality of state-machines. 3.3.1 Limitations of state-machines Before defining a more enhanced state-machine we should discuss their limitations. The one-way-limitation Figure 3.3: A one-way state-machine A state-machine allows only one way of applying the transitions. There is no way to do parallel transitions. It would be better to specify the correct sequence of calls, but sometimes it is not possible to specify only one allowed way. Figure 3.3 shows a sequence of transitions that result in state Y. Reaching state Y in this example is only possible going from state A to state B, etc. 1 The page number referes to the german version.
  • CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 21 Sometimes a state should be reached when several other states have been reached without defining an order. State-machines that allow such sequences will quickly get too complex to handle. Figure 3.4: State-machine for permutations of calls Defining groups of states One solution to this problem could be building groups of states, but state-machines do not support groups. There must be policies which transitions are valid within the group and which cannot be applied. These policies will get less complex if they can be defined for state-machines as in figure 3.5. Figure 3.5: State-machine similar to a one-way state-machine In this state-machine all transitions that result in a new state can be called in this state again.
  • CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 22 Components which can be modeled by such state-machines consist of methods that can be called at least twice without causing an error. This might not apply for all methods but a lot of methods show this behavior. For instance, during the setup of a component it might be necessary to store references of other components by calling a method. This method can be called several times without causing an error. An example for a method that does not have such a behavior could be the opening of a file containing the setup information. Calling this method twice may cause an exception when the file is still opened. A robust implementation is to avoid such an exception and to make it possible to call the method more than once. 3.3.2 Enhanced state-machines Enhanced state-machines allow to test contracts as defined on page 19 by using a set of states. It supports a group of states. All transitions that are allowed between these states can be applied. A method that causes a switch to a new state but can not be called again in this state, can also be called once in these state-machines. After a state-machine has been defined, this enhanced functionality can be applied by grouping states. Figure 3.6: An enhanced state-machine 3.4 The categorization of contracts There are various categories of contracts. For implementing systems that deal with contracts it is important to define categories for these types. Some types do not need
  • CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 23 additional algorithms but others need loop-detectors or other algorithms based on graph-theory. 3.4.1 Linear Contracts Linear contracts are the simplest type. Each state has only one transition that ends in another state. Figure 3.7: Linear contract It is very simple to handle such contracts, as there is a clear way of how a component that supports this type of contracts has to be used. 3.4.2 Pseudo Linear Contracts This type is slightly more complex than the linear type. One or more states have one or more transitions that end either in the same state or in a state that has not been visited. Figure 3.8: A pseudo linear sequence Building a system that is able to handle such contracts must solve two problems:
  • CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 24 • There are different ways of reaching a state. • How often is a transition called that ends in the same state. 3.4.3 Looped Contracts Developers should avoid writing contracts that have this behavior, but sometimes it is necessary for a component to have such a contract. Figure 3.9: A looped sequence A system for these contracts must solve several problems: • There are different ways of reaching a state. • How often is a transition called that ends in the same state. • How to detect loops. • Finding all ways to reach a state. • Finding the shortest way to reach a state.
  • CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 25 3.4.4 States and pre-conditions Most models for testing software components use pre-conditions to define what is to be done before a component can be used. Pre-conditions are often defined in a very formal way resulting in lots of work to define them. States are defined during the design of a component. Using this view on components may result in a simple design and simple designs may cause less errors. In some way there is a relation between states and preconditions: • Methods that need the same pre-conditions are in the same state. • The summary of all states that are needed to reach a state can be interpreted as the pre-conditions of this state. It it seems possible to derivate most pre-conditions by using states. However, this has not been proven and evaluating this assumption and developing an algorithm should be part of further research. 3.4.5 How to combine components Components will be used in combination with other components. All these com- ponents have contracts that must be fulfilled. The components are connected by applying method-calls on each other. The component that applies a method-call on another component is called caller. As described in section 3.1.3 there are different types of method calls, determin- istic calls and nondeterministic calls. Figure 3.10 shows the model of combining components by applying deterministic calls. This is quite simple because invoking a deterministic call will always result in an end-state. Image 3.10 shows the model of a deterministic call between two compo- nents. Both involved components change into their end-states.
  • CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 26 Figure 3.10: A call of a deterministic method Figure 3.11: A call of a nondeterministic method Calling a nondeterministic method is more complex because there is more than one possible end-state. Both components must handle all possible exceptions and switch into the corresponding state (cf. 3.11).
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 27 Chapter 4 General software quality factors Start by doing what is necessary,then do what is possible, and suddenly you are doing the impossible. St. Francis of Assisi Software-quality is a controversial topic as creating applications are getting more and more complex. Developers have to manage the trade-off between fast development cycles in a complex environment and customers that need stable, reliable products. The situation is getting worse when applications are used by people that do not have a technical background. Technicians often find solutions that are logical and stable when they use them, as they know the processes in the background, the relations between them and the resulting operation of a certain device. People who do not have this background information - because it is not their main topic and they only want to use a device or an application - often get problems with these products because their usage is not intuitive and resulting bugs do not allow an efficient usage of the product. The problem of a not intuitive user-interface or behavior of a product can only be solved by usability tests during the design. Decreasing the rate of errors can be done during implementation, but the basis must also be established during the design.
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 28 It is common sense that it is impossible to write huge applications without bugs, but this should still be the goal during the development process. Developers may not lean back without struggling for reaching this goal because everybody says that it is not possible to reach it. Economical conditions force companies to decrease the development time and its costs but this is short-sighted because bugs in deliv- ered applications cause much bigger costs than bugs that are detected during the development process. It doesn’t matter if dishwasher, car, washing machine or mobile phone - if a device doesn’t work, there often is a bug in the software. [...] Experts quite agree that software could be essentially more reliable if producers would force that. Further, they believe that a liability for bugs could speed this process up. Annually buggy software costs 60 bn. USD for the US-economy [...] There are no automatic test-tools1 : One half of their time developers devote to programming, the other one to searching bugs and patching these. In the test phase, a program reveals approximately 10 bugs per 1000 lines of code. The problem of developers consists in that there exists no robust tool to measure the reliability of their software-designs. [ORF03] Users have accepted that an application crashes once a day, not because they know about the complexity of the programm but for the reason almost every programm behaves in that way. As a consequence they have accepted this misbehavior as a bothering, unavoidable fact. As our life is getting more and more digitalized, more reliable software is needed. In the IT-world there is a lack of experts, so lots of projects are delegated to non-ex- 1 The term “automatic test-tools” does not mean that they can automatically generate test- cases, but they can automatically run test-cases. Automatically generated test-cases would never be as efficient as test-cases, manually created by intelligent inspection of source-code.
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 29 perts resulting in faulty products. By means of new and more powerful programming languages, tools for testing and formal specifications, the IT-community is trying to remedy this situation. This is not the final solution, but certainly a step into the right direction. One of the greatest “revolutions” in software design during the last few years has been both, the invention of the object-oriented approach and the introduction of the OOP quality factors, like reusability, modularity and extendibility (section 4.2 on page 33). The evolution of OOP temporarily ended in the definition of the Design Patterns which can be interpreted as the object-oriented counterparts of the established paradigms of structural languages. These patterns allow the implementation of components to fulfill the goals of the OOP approach. Christopher Alexander provides a good description of patterns: “Each pattern describes a problem which occurs over and over again in our environments, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice” [GHRV95, Chap. 1] This definition describes patterns in buildings and towns, but it can also be applied to OOP. Patterns define solutions for general problems using a range of known patterns in a very formal way. A very good book by Gamma, Helm, Johnson and Vlissides (GOF ) is Design Patterns, Elements of reusable object-orientated software. Section 1.1 [GHRV95, p.2] defines the term pattern by specifying four elements: the pattern name, the problem, a solution and its consequences. The book addresses 23 patterns that are grouped into 3 categories. All patterns are explained and further treated by discussing an example, so that it is easy to get an overview of all patterns. Knowing these patterns is not enough, they must be internalized to understand their deeper meanings and their potential. These 23 patterns are patterns for object-orientated development environments.
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 30 As the development-environment provides special possibilities, e.g. the J2EE2 - platform, existing patterns will be adapted3 or new ones will be invented4 (see [AM01]). In combination with the ideas of OOP a further definition is becoming popular, namely that of a component [Sch02] A component is represented by a class that concentrates on one topic and which is connected with other components via a Hook and a HookUp (see page 40 for details). Another aspect of modern languages is that they are very useful for writing correct and robust programs as they provide the chance to handle unexpected situations by defining a clear mechanism for notifying such situations. This can be done by defining exceptions which track faulty states of a program and guide the program to a defined end- or error-state. The aim of this chapter is to explain the terms which are needed in the following chapters including design patterns and software quality factors by means of a simple example5 . 4.1 A small example The example is implemented as a component-based framework6 allows to build string manipulation applications. The framework captures the main design and thus as- signs the responsibilities. It therefore dictates the architecture and allows the devel- oper to concentrate on the specific requirements of the concrete application which is 2 A good book on J2EE is Mastering Enterprise JavaBeans by Ed Roman [Rom02] 3 The command pattern has been adapted to J2EE see [AM01, chapter 7.2]. 4 The business delegate[AM01, chapter 8.1] was designed especially for J2EE 5 The entire example and the usage of CrashIt as its test bench can be found in the appendix. 6 A framework is a set of cooperating classes that are designed to support developers in creating applications for different domains [GHRV95, p. 26].
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 31 based on the framework. This framework is designed to build string manipulation applications. The framework defines Services that can be used by a ServiceProvider. A Service deals with strings on which it performs some computations by invoking the method String execute(String value). The framework consists of several classes, of which two are important for the further explanations: Service: it processes a string value and returns the modified value. A Service must be initialized, before using it. It is possible to check whether the service is initialized or not. A Service has a unique id, that serves ServiceProvider to address the this Service. ServiceProvider: it manages different Services. The ServiceProvider7 can ini- tialize all registered services. A service can be used by passing two parameters to the ServiceProvider, the id of the service and the value that should be processed. 4.1.1 Interfaces There are different interfaces that correspond to the parts of this framework. In this section the interfaces are shortly described a detailed documentation can be found in Appendix A.2.1 on page 89. The service interface (see listing A.1) consists of four methods: • String getServiceId(): returns the id of the concrete Service • String execute(String) throws ServiceException: performs a calcula- tion on a value • void init() throws ServiceException: initializes the service 7 The ServiceProvider can be interpreted as Mediator [GHRV95, Chap. 5].
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 32 • boolean isInitialized(): returns true if the service has been initialized The service-provider interface (see listing A.2) is defined by8 : • void registerService(Service) throws ServiceException: registers an uninitialized Service • void initServices() throws ServiceException: initializes all services • String ask(String id, String value) throws ServiceException: forces the service that corresponds to the id to process the assigned value 4.1.2 Implementations and usage of the framework In this example two simple services are implemented: Echo: returns the given value Concat: concatenates the given value with the previously stored values (The first time Concat is called, it returns the unchanged value.) 9 These two services can consequently be registered at the ServiceProvider and addressed by it. This can be done by implementing the following steps: 1. an instance of a ServiceProvider must be created 2. both services are loaded 3. an external object registers the services with the provider and 4. the provider initializes these registered services 8 A smarter implementation of this components would go beyond the scope of this example, but would be necessary for real world applications. 9 This implementation is done in the class org.service.ServiceProviderImpl that is part of the CrashIt example. The whole example is part of the CrashIt distribution
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 33 5. the services can be used by applying the corresponding key and a value An example of an application that uses the framework in the described way can be found in Listing A.4 on page 95. Services and ServiceProvider will be connected at runtime which allows a very flexible configuration and extension of an application that uses this framework. If this application must be modified, only its configuration has to be updated without recompiling the framework. For instance, the implemen- tation used in the appendix supports a Configurator class. It is responsible for setting up the ServiceProvider. Configurator parses a configuration file, where the used services are specified. In this version services can be added during the start up. A smarter implemen- tation would be able to add services during execution, but this example is designed to clear the view on the basic ideas and not to provide a perfect solution to the problem. Before starting to explain the remaining terms, the following sections deal with achievable software quality factors. 4.2 Important Quality Factors The premises, that must be fulfilled to produce high quality software which the economy demands, can be grouped into these external factors : 1. Correctness is the exact representation of a software product with respect to its specification. 2. Robustness is the ability of software to respond appropriately to abnormal or modified conditions. 3. Extendibility is the ease of extending software products to new changes of specification or other problem domains.
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 34 4. Compatibility is the ease of combining software elements with others. 5. Ease of use is the ease with which people of various backgrounds and qualifi- cations can learn to use software products and apply them to solve problems. 6. Performance is the capability to predict the time exposure of an applica- tion depending from the software architecture, amount of processed data and hardware and in following internal factors: 1. Simplicity is the ability to model real world problems in intuitive software structures. [Bec00, Chap 17] 2. Modularity is the ability to decompose an application in single modules, which communicates only over defined interface. [Sch, Chap 2.4] 3. Intuitivity is the capability to write the code so that other developers may read it like a book, this without comments in the code. [Sch01] External factors define characteristics of a program determined by users, whereas internal factors concern software engineers. They have access to the software design and its sourcecode. The following sections explains in detail these factors. Further external and internal factors are explained by [Mey97], [Ale01] and [McC77]. 4.2.1 External factors The journalist of the article [ORF03] tries to picture out, why many software pro- ducts are not reliable (neither correct, nor robust). As an outstanding user, he judges the actual situation in the software industry. This judgment from external users equals the definition of the external factors. In this sections the most important external factors with respect to CrashIt will be discussed.
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 35 Correctness Correctness is the main and prime quality of a software product; no other factor has such a weight like this. The best user interface counts nothing if the software is not correct. The correctness is theoretically derived from the exact implementation of the URD (User Requirements Document). But also the URD must underly correctness and completeness. For little systems an exact URD is possible, but for huge applications like an operating system, the numerous user requirements for the software product could easily conflict among themselves. To create reliable software, the correctness already starts when user requirements have to be declared and verified on the client’s desires. Correctness finishes with the maintenance of the product. A method which helps to employ correctness is the layered structure of software: The software can be split into more clearly defined and limited layers. Each layer relies on lower ones. The process to proof the correctness of a layered system is a incremental process. That means that the correctness of a layer depends on the correctness of lower layers. If, for example, a system makes use of libraries then firstly it must be ensured that these are correctly implemented. Relying on this as- sumption it is possible to test the correctness of the system. The basic layer should be selected evaluating user requirements, operating system, language and compiler, hardware, running environment and user groups. Design patterns may help to find the layers and interfaces between them. [Hoa72], [BBM+ 78] Robustness Correctness is the exact representation of the specification, whereas robustness also expresses how a system reacts in situations which are not include in the specification. The strength of robustness consists of the capability to notice failures, to guide the system into a secure state or to catch the error and start some recovery processes. To grant robustness, following points help:
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 36 • Clear and simple design: thereby it is easier e.g. to locate errors and to react accordingly • Formal specifications, see [Jon96] • Start testing already during specification phase • Not only valid tests should be formulated, but also invalid tests, which should verify the robustness Extendibility An essential principle for improving extendibility is modularity: The better a system can be split into autonomous parts, the higher the likelihood that a change in the system would only affect one or some modules, rather than the whole system (further details are explained in section Modularity on page 39). Compatibility A commonly known situation: A new operating system is installed and some old applications would not run anymore. This is a common issue of not considered compatibility. Compatibility is the factor which implies the most work: Depending on the size and differences of a user group, this factor demands to consider each different environment, where the user would use the new application. Also for this factor, layered program structure could help to move all user or system relevant varieties into one or less layers. Using this technique, only some less layers would be affected by compatibility. Of course, if the specification is not good enough, the migration to other platforms is quite impossible. Ease of use The factor ease of use should just be considered when user requirements are to be defined. The more software designers understand and study the desires, backgrounds
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 37 and living environments of the user group, the more user requirements can be ex- pressed through a simple and clear structure. Upon user requirements, the software design is developed. The simpler and clearer these requirements are, the easier the process of development becomes and the more uncomplicated and unproblematic the structure of software can be defined. Simplicity and clarity are premises for this factor. Performance This important factor would be ignored, if for example the calculation of tomorrow’s weather takes more than 24 hours. Mostly, the performance of a system correlates with the hardware capabilities, but also with the complexity of a program and how much data types will be abstracted (The abstraction of types means its generaliza- tion. This has the disadvantage, that special types must be derived from general ones. This process causes performance costs). The software community shows two typical attitudes: • Either a program is designed with respect to speed omitting intuitiveness and simple design (this program would be so specialized, that it is not open for reusability or extendibility) • Or it is designed with respect to generality: design a program as general as possible to be used for many different problem domains, without dealing with process time or other limited resources. 4.2.2 Internal Factors Internal factors describe qualities for software designers. These factors are in general the premise for external factors: correctness, robustness, extendibility, compatibility and ease of use based on simplicity and modularity.
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 38 Simplicity Simplicity could be one of the hardest things in peoples’ lives: It seems crazy, but sometimes it is easier to do something more complicated than to do things simple. This is particularly true, when you have been suc- cessful doing the complicated thing in the past. Learning to see the world in the simplest possible terms is a skill and a challenge. The challenge is that you may have to change your value system. [Bec00, Chap 24] Most people correlate quality work to simplicity. For example a car: it has few com- mand devices, but they are usefully arranged, so that driving is a simple task. The manufacturers of cars hide complex structured engines behind simple to use devices. A gimmick to achieve simplicity in software: first avoid complexness, secondly hide complex processes behind simple-to-use devices. Simplicity will not only influence the external factors, but also: • A simple design is easier to communicate with than a harder one. • Simplicity invokes user to adopt simple strategies to achieve simple and fur- thermore also correct, robust and extendible programs • Simple design gives quicker, more and exact feedback about the quality of the project. Principles to improve simplicity are: • develop software using a modular approach and Design Patterns (see [GHRV95]) • small initial investment in the design • incremental change: The strategy of simplicity in design will work by gradual change • be open for extensions and changes in the specification
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 39 • ability to understand user requirements, their background and living environ- ment. Only on this way, software developers are able to apply correctly user desired software solutions. A further key to support simplicity is uniformity in the programming style (e.g. using some style guides). Another key is of course intuitivity, so that a concept of a system is understandable and logic for colleagues in the developing team. Finally each developer should keep in mind, that a system has to be maintained. It is therefore useful to get the habit to write exact documentation, furthermore to think and develop in simple terms. Modularity A software construction method is modular, then, if it helps designers pro- duce software systems made of autonomous elements connected by a coherent, simple structure. [Mey97, Chap. 3] A modular design method could be classified using this five criterions: • Modular decomposability: a system can be split into less complex and rela- tively autonomous parts that communicate over a simple structure • Modular composability, is connected to reusability: it represents the capability to construct new systems using existing modules. Only the communication structure has to be designed. • Modular understandability: each module can be read and understood by itself, without knowing other or only few modules. • Modular Continuity: an extension in a module does not trigger changes in other modules. • Modular Protection: the consequences of an error caught or caused by a mo- dule are limited to that module or at worst propagate to few neighboring modules.
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 40 4.3 Design patterns Design pattern describes a problem, that often occurs in our environment, and gives a solution to this problem (see also chapter 4 on page 28). Design patterns often seem to be very similar or they can be described in similar ways, however, their small differences are quite important. Patterns are sometimes based on related concepts which is the reason why their deeper meaning can only be caught by studying their application and its resulted consequences. As carefully worded definitions of the patterns can be found in the design pattern book [GHRV95]. In this document only a few patterns are used, the most important one in this scope will be discussed in the next section. It is a basic pattern that is often used although it is not explicitly mentioned. Further readings about Design Patterns is well documented in [Ale01] and [Sut00]. 4.3.1 Hook and HookUp The Hook Hook-Up pattern (refer to [Sch02]) is split up in two parts, the Hook and the HookUp (see following subsections). This pattern is often used unconsciously by other patterns like the Observer pattern (see [GHRV95]). The deeper intention of the Hook / Hook-Up pattern is to connect components. Moreover, it is not described in detail in the Design Patterns Book [GHRV95] because it was later defined by Klaus Schmaranz, [Sch02]. Some patterns use the hook pattern for connecting components, but depending on the behavior of the connected components these patterns describe different func- tionalities. For example, the Observer pattern combines components by using the hook pattern but in contrast to a Mediator the observer notifies all connected components when an event occurred. The mediator that also uses the hook mechanism for connecting the components, chooses only one component to forward the event.
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 41 There are some other patterns that use the hook mechanism, but it is not the aim of this document to explain all design patterns, however, for a better understanding of the next sections it is useful, to have a closer look on the hook pattern. Hook The definition of a Hook consists of two parts: the abstract Hook and the concrete Hook. The abstract Hook defines an interface for a component that represents me- thods (e.g. execute) how this component can be used. A concrete Hook implements this abstract Hook interface. The Hook-method is the “reason” why a class is written. It contains the functiona- lity that is needed. If more than one Hook method is needed, several abstract Hook interfaces must be defined, to make clear how this component can be used. In the service-example (section A.2 on page 88) the Service interface is an abstract Hook-Up and the Echo class is a concrete hook because of the execute method that is used to ask the service. Hook-Up The Hook-Up is the second part of the Hook Hook-Up pattern. In comparison with the Hook that defines what happens when a method is called, the Hook-Up decides when the method is invoked. The Hook-Up can also be split up into two parts: the abstract Hook-Up and the concrete Hook-Up. An abstract Hook-Up defines a method that is used to connect a Hook with another component. A concrete Hook-Up is a component that imple- ments the abstract Hook-Up interface. In the example the ServiceProvider (Listing A.2 on page 92) the interface is an abstract Hook-Up because it contains the Hook-Up method register(Service). The ServiceProviderImpl class is a concrete Hook-up.
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 42 4.3.2 Components The idea of components is based on the main concept of object-oriented program- ming. [Sch02] • A component is responsible for one function or can deal with one type of data. • A component must be free of side-effects. • The communication with a component is only allowed through methods, public member-variables are not admitted. • There are only a few ways how this class can be used. Because of these constraints components are able to signal missusage. Mis-usage can only occur by applying wrong values to methods or by calling methods in a wrong sequence. Developers often violate these ideas which are also an important part of the OOP approach, so the stronger term component was defined to make clear, that if someone is writing a component, this class complies with these ideas and limitations. In the service example the different services are typical components. Some guidelines how components should be designed will now be discussed. These guidelines widely correlate with guidelines for designing exchangeable and expand- able components, and they also force the design of good testable applications. So complying with them, these guidelines are not a restriction to the freedom of software-designers but rather implicate a good design for an exchangeable and ex- pandable component-framework. Side-effect-free : A component must avoid side effects. It should encapsulate the functionality without affecting other components (e.g. changing global variables), except those components that are registered in order to receive reactions from the component.
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 43 Composition by Hook and Hook-Up : The Hook Hook-Up pattern must be used for binding components [Sch, Chap. 3.4.7]. This includes the requirement for defined interfaces for each Hook and Hook-Up. The interfaces describe how a component is registered and which methods can be used. The Hook Hook-Up pattern allows test teams to connect components via a proxy that can be used for monitoring the test. This can be done dynamically without recompiling the component. Dynamically loadable : This correlates with the guideline of using the Hook Hook-Up pattern. Components must be dynamically loadable in order to replace them without recompiling the framework. Moreover, this allows the usage of components that are not known at compile-time. [Mey97, Chap. 2] Thinking of the inner state of a component : During the design process the inner state of a component should always be present. The inner state of a component will be changed by calling a method. If the method finishes correctly, the next state is reached. If an error occurs during the execution, another state is reached. Thinking of the effects of method-calls will help to design components with small interfaces. Usage of exceptions : Exceptions have to be triggered on unpredictable events or mis-usages of a component. Even wrong method-calls should trigger excep- tions because a programmer is not forced to handle special return-values. If a method returns true when it succeeds and false when it fails, the programmer can ignore this result. If an exception is thrown when it fails, the program- mer must handle it. Using exceptions will also force designers to think about the handling of missusage or wrong method calls during the design. Thinking about the impact of an exception on the inner state of a component during the design-phase facilitates a robust component design. Notice that it is not allowed to use exceptions for controlling program flow because, this would vi- olate the concept of an exception. Exceptions are not designed to handle the default cases but to indicate dangerous situations (see section 4.4 on page 44).
  • CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 44 Definition of the usage : Components must be defined with respect to the con- text, where they are used. Therefore, obligations and benefits with reference to the context can be defined. This definition can also be called contract. For more details about contracts refer to [Val03, chapter 4, p.34]. 4.4 Exceptions Even when all components are correctly used difficulties can occur, e.g. an inter- rupted network connection. Program should be able to manage these situations. Exceptions are the correct tool to solve such problems. An exception can be interpreted as a possible result of a method call that occurs only when something goes wrong. This is an important aspect because exceptions are thrown when a dangerous situation has been reached, whereas they must not be used for controlling the program flow. If the program flow depends on a result of a method call, the result value should be used and not a possible thrown exception. Several languages implement the exception-concept in different ways. Java, for example supports exceptions as a part of the language concept by providing special keywords to handle these situations. All implementations allow the definition of an additional program code that is executed when a dangerous situation has been signaled by throwing an exception. Notice that for the test design an exception can be a valid test result. It is necessary to test if an exception will occur caused by missusage or an unexpected event in the application. The emergence of an exception is part of the component behavior.
  • CHAPTER 5. TESTING 45 Chapter 5 Testing The English Positivist philosophers Locke, Berkley, and Hume said that anything that can’t be measured doesn’t exist. When it comes to code, I agree with them completely, Software features that can’t be demonstrated by automated tests simply don’t exist. Kent Beck Another philosophic approach to testing can be found by interpreting articles by Sir Karl Popper. He defines the matter of a scientific thesis or a scientific work by two factors, the logical and the empirical matter [Pop94, S 40]. The logical matter combines the sum of all works that can be derived from a thesis. This can be interpreted as the reusablility of a component. On the other hand, the empirical matter of a work is defined by the sum of all proves that demonstrate the correctness of a thesis. This second factor can be interpreted as the testability of a component. During the last few years patterns and components have become popular so more and more papers or books about these “solutions” can be found in nearly every software design division. As a consequence we claim that the evolution of writing reusable and expendable software components by using the ideas of the design patterns is nearly finished. We can say that the IT community has learned to carry out the
  • CHAPTER 5. TESTING 46 ideas of reusable and extendable components and is able to fulfill the goals of the OOP approach. So the logical matter of components that define an application can now be satisfied. But what about the second factor? The aim of this section is to discuss different test concepts. It will also show how the component-based approach which has been described in earlier sections, allows to define tests and thus it facilitates to increase the empirical matter. How can this be done and what are the steps to achieve this goal? 5.1 Goals for running tests In this scope another aspect become more important: Do these reusable components react as it has been defined during the design? At this point new terms gain impor- tance correctness and robustness. The target of a test is to check implementations against these terms. If someone is in the situation that the test of a new software component must be defined, there are two frequently asked questions: • What are the goals of this test? • Does this test verify the correctness of a component? However, these questions are dangerous because they imply that a test is carried to show the correctness of a program, which is wrong! A test is done in order to find failures or misbehaviors of a program and not to show that everything functions correctly. This psychological aspect of testing is one of the reasons why testing is not as popular as designing. Testing is a destructive task that charges the components in order to affect them to cause errors. It cannot be done by using the default test-cases,as they will result in default test results. Instead it must be done by using test-cases that are more complex than the default ones and
  • CHAPTER 5. TESTING 47 this must be carried out in an environment that is close to the real-life situations in which the component will be used. A typical situation is that after a test someone asks: “...ok, this was the test, but did it fail or succeed?” Such situations must be avoided! It must be clear how a result of a test run has to be interpreted before the test is done. It is therefore helpful to use tools that are able to generate clear summaries of what has happened during the test. At least almost all scenarios that are discussed in this work deal with the problem of verifying the correctness and robustness of a program, but there are other possible targets too. 5.1.1 Targets of test runs Keeping the main goal of finding errors in mind other useful targets can be defined for the test. For all targets it must be clear that a test is done to find misbehaviors and not to show that “everything is ok” in order to ease the development team’s conscience. It is not the aim of this work to give a complete introduction into all possible test scenarios, but it is necessary to keep all requirements of a software project in mind when the tests are designed. It is likely that not all requirements of a project can be proved by only showing the correctness and robustness of a program and therefore testing this aspects is an important but not sufficient part of the whole test process. What are possible goals for test runs? Correctness-tests : Testing the correctness of an application or a component can only be done when its behavior is known and well-documented. The correctness of a component can be tested by applying method calls in the correct order on the component. These calls must also use correct parameters so that the component is used correctly. If there are different ways how the component can be used, all ways must be included in the test. For example, if a class can be instantiated by using different constructors, all these constructors
  • CHAPTER 5. TESTING 48 must be used in test-cases. That applies to methods with different signatures as well. This might imply a big effort but it is necessary to guarantee that all possible ways of using a component are included in the test. Robustness-tests : In contrast to testing the correctness of a component testing the robustness uses incorrect calls i.e. calls with incomplete or illegal pa- rameters or invokes methods in wrong orders. This test concentrates on the feature of the tested component that a misusage is detected and reported. The component should neither cause a crash when it is misused. Even if testing the correctness implies a big effort when all correct ways of using a component should be included in the test, testing all ways of its misusage is nearly impossible. The only way to reduce possible missuses of a component is a clear and simple design, which will also decrease the effort for testing the component for robustness. For testing the robustness it is necessary to accept exceptions as desired an- swers of a illegal method calls. Consequently, testing the correctness and the robustness can often be done in the same test run when the result of a call is only compared to the desired results. Performance-tests : Testing the performance of an application always means comparing its behavior with target values. It is necessary to carry out a test with a multiple of the expected critical actions. Critical actions can be network requests, user interactions, queries etc. Notice that the test should be done with at least 7-10 times of critical actions to ensure that the application reaches the expected performance. This performance test must be done on an appropriate computer system which is pretty similar to the system used for running the application. The performance-tests can deal with different aspects: • time behavior : This deals with questions like: “How long does it take to execute a request?” or ”How long does it take the application to start
  • CHAPTER 5. TESTING 49 up?” A related topic is the monitoring of the response time , i.e. the time that an application needs to return the first partial answer. • resource consumption: Nearly every application is forced to reduce the consumption of resources. There are different types of resources. One of the most important ones is the memory usage of the application. Me- mory is limited, so an application should optimize the amount of required memory. If a program uses more memory than it should, this often has a bad influence on the time behavior, because the missing memory must be simulated by using the much slower hard disk space. Other resources are limited by harder system-depending variables. For instance, operating systems often allows only a limited amount of open file handles or possible network connections. If an application tries to exceed this limits, this will often cause unsafe situations for the application, if nobody considers this situation during the design. A third group of limited resources also causes a deterioration of the per- formance. For example, database connections are also often limited but depending on the complexity of the executed queries the upper limit of connections varies. Testing an application with a huge amount of database connections that perform simple queries can pretend a good performance but in real life the application uses more complex queries and the performance will drop radically. A consequence, the test must be done with typical use cases, especially when more complex subsystems whose behavior can not be described very well are included. Interoperability : Interoperability is necessary when more than one system ar- chitecture is used. Different byte orders on the network layer or different implementations of used protocols may cause errors that can hardly be found. An other important case for testing interoperability occurs when a new stan- dard is implemented by different groups for the first time. Interoperability-tests must always be carried out by combining the different implementations in a way that all possible combinations of the related com-
  • CHAPTER 5. TESTING 50 ponents are tested. This permutation can be skipped when a reference imple- mentation is available. There are several other possible test scenarios, but this document concentrates on correctness and robustness, so discussing other factors would go beyond its scope. Other possible test scenarios can be derived from the requirements of the application. In each case it is necessary to think about the goals of a test run and write them down. After this part has been finished the test will be designed and implemented, which often causes a big effort, sometimes similar to the design. It is useful to start the test design and its realization during the design phase to avoid untestable modules. In eXtreme Programming [Bec00, Chap. 9] the definition of the test is the last step before the implementation is started. This confirms the importance of testing in the design cycle. Thinking of how a component can be tested during the design often facilitates a modular and extendable design. 5.2 Test-design concepts Defining good test-cases is often hard work and test engineers need a lot of infor- mation about the program to find all important test-cases. The component-based approach simplifies this process because a lot of test-cases can easily be found by studying the descriptions of the components. The component-based approach is based on a detailed description of the interfaces between the components. Without this description it is not possible to write or later use components, so designers are forced to write down how components must be used and how this components must react. Pre- and postconditions and a sequence of how methods had to be invoked are part of the documentation and facilitate the process of defining the test-tasks.
  • CHAPTER 5. TESTING 51 The component-based approach implies the development of good testable programs, because each component interacts with other components only by using a defined interface. This is only true when the system is split into quite small components that interact in a comprehensible way, but this is a requirement of the component-based approach. If the component-based approach is used, an other big problem is solved: the design of an integration test. Testing the whole system is often quite complex as the system often hides information and components cannot be accessed during the test. The component-based approach allows to use proxies between the components during the test and so the communication between the involved components can be monitored. This is very useful to determine if a component is misused or wrongly implemented. Even when proxies are used it can be very complicated to determine what went wrong, so the test must be a step-by-step procedure. 5.2.1 Test principles There are some simple principles how component-based systems can be tested. The main idea is to define tests from a simple to a more complex scope. Here are some rules how tests should be designed. These simple rules seem obvious, but they define a method that can be used when a test must be defined by hand1 . 1. Test Components on their own: First all components that could be used with- out other components should be tested. In the service example the two services Echo and Concat should be tested on their own in order to verify that their reacts corresponds the specifications. The goal for this test is to check if the component is implemented correctly. 2. Combine few components via proxies: As a next step, the test-engineer should combine only few components (mostly denoted as Client and Supplier) 1 In later versions of CrashIt there will be an automated way to generate test configurations using this approach
  • CHAPTER 5. TESTING 52 via a proxy. This test is useful to check if components are used correctly by its Hook-Up component. The proxy allows the engineer to monitor the com- munication between the Hook-Up and the Hooked component. For instance, CrashIt supports these proxies but they are called Contracts(refer to [Val03, p.59] for detailed information). Contracts are slightly smarter than simple proxies because they can interact with each other. In the service example this test can be implemented using the class org.service.ServiceContract. 3. Combine more components directly and new ones via Contracts: If the test has been successful so far, more components can be combined to a more complex system. Components that had already been tested by using Contracts can now be connected directly. New components should be connected by using Contracts between the client and the supplier. The reason for this test is to check if a growing framework reacts as defined in the specification. If a problem occurs, it could be useful to connect components that had already been tested again by using Contracts. This allows to gain more information on what has happened during the test. The reason why already tested components should be connected directly is that the system reacts in a way which is more similar to how it will react in a real world environment. This can be falsified for example by the delay caused by a Contract, so only the important Contracts should be used in this stage. This test cannot be carried in the mentioned example (see section 4.1 on page 30) as there not enough components. 4. Test the whole system: After all these tests have been passed a test of the whole system should be done. Now the real performance and delays of an event can be determined. In the small service example this is done by using the configurator. The configurator connects the components directly without Contracts. Sometimes it can be useful to connect components via Contracts even at this stage in order to trace the communication between the compo- nents, for example to determine components that have a poor performance.
  • CHAPTER 5. TESTING 53 Developing test configurations by using this simple schema is comprehensible and configurations can be easily enhanced by including new components or new test cases without dropping the old configuration. This schema can be used in all test tools that are able to deal with components. In jUnit[jUn03] such configurations can be implemented by writing test-suits that combine several test classes. 5.2.2 Descriptive or declarative configurations Writing the test-code is not the only way to carry out tests. It is also possible to use external files to define such test configurations instead of implementing test-classes. This has the advantage that these test scenarios can be designed and written before one class has been implemented. The test team can start their work while the development team is working on the implementation. The approach of writing files that describe the test sequence avoids more errors in the test configuration than writing a test-class, because describing what should be done is more simple than implementing the test-case. Implemented test-cases often contain copy-and-paste errors in the source code and its error-rate is as high as in the source code of the program. Therefore it is useful to change the medium by using different formats for defining the test-cases and implementing the programm. Another advantage of describing the test scenario in a language-independent format is the possibility of using the same test-cases to test another implementation based on another language or environment. For example, the configuration can be used for testing Java component-based frameworks and another test-bench can use the same configuration for testing an C++ implementation. An implementation of a test-class may be slightly more flexible than the description of a test-case, because only topics can be tested that can be described in the used description. A small set of needed operations and definitions is enough for testing
  • CHAPTER 5. TESTING 54 component-based frameworks. If this method is applied to a framework that has a clean component-based design this disadvantage loses importance. 5.3 Exception safety and robustness As mentioned before exceptions are an important tool for increasing the robustness of a program. Even when a program is written without exceptions critical situations may appear. Without exceptions these situations have to be indicated by a special return value. No one forces the programmer to handle this special return values and they are often ignored. If a situation is indicated by an exception, the programmer must handle it because otherwise a compile error will occur. Even when exceptions are used in a program, it is still the decision of the programmer how an exception must be handled. When an exception is ignored this is called silent catch. In some uncommon cases these silent catches are necessary, but in the majority of cases they are programming mistakes and they therefore have to be avoided. This is only one reason why it is necessary to test if a component throws an exception when it is supposed to. Testing the robustness of a component can often be done by testing if an exception occurs when the component is used in a wrong way. It is useful to use only a small number of different exceptions that may occur during a method call because handling a lot of different exceptions on their own may cause a big effort. In a layered architecture for example, the application layer should not deal with low level exceptions. Each level should define and handle its own exceptions. Java offers the option of wrapping an exception within another by using a pre-defined constructor of the Java exception object. Sometimes it is useful to use these nested exceptions 2 to forward an exception through different API levels that use different exception types, but generally each level should handle its exceptions on its own to 2 Compare to the chain of responsibility [GHRV95, p. 223]. Nested exception are nor equal to a chain of responsibility but it is a related concept.
  • CHAPTER 5. TESTING 55 provide a transparent usage of this level even when an exception occurs. However, despite all their advantages exceptions are not as widely used as they should. Programmers think that it is inconvenient to be forced to think of critical situations during the implementation and the design but it is necessary. Even when exceptions are not so popular let alone testing them. A good test concept must also cover exception testing. The purpose of the last chapters was to motivate the topic of testing in a theoretical way. The following ones will discuss this important topic from another point of view.
  • CHAPTER 6. CRASHIT - A SHORT INTRODUCTION 56 Chapter 6 CrashIt - A short introduction It is not enough to be busy; so are the ants. The question is: What are we busy about? Henry David Thoreau Almost every developer implements tests for the written code to verify it. Sometimes a framework like JUnit is used to implement these tests. This enhances the potential of the test code because it can be started automatically and it can therefore be shared between different programmers who do not know details of the components that will be verified by running these tests. As JUnit uses implemented test-classes it is a declarative approach. The framework which was designed and implemented during this work uses a descriptive approach. It is called CrashIt . The aim of this chapter1 is to give a short overview of CrashIt its concepts and its usage. A full introduction into CrashIt can be found in [Val03, p. 41]. 1 This chapter includes some passages of the whole chapter in [Val03, p. 41]
  • CHAPTER 6. CRASHIT - A SHORT INTRODUCTION 57 6.1 What is CrashIt ? CrashIt is a framework which supports developers and test-engineers during the unit- and the integration-test of software components. The aim of the project is to motivate developers to write test-cases based on their specifications. These test- cases will be processed by this framework, which finally returns a report about the whole test. A test2 of a component-based system consists of 1. checking the behavior of each component (= component-test or unit-test) and 2. monitoring the communication between components in order to detect mis- usages and check the behavior of components in a network (= integration- test). The framework actually provides the capabilities to • define tests (test-cases, test-case-sequences, configuration), • run tests and log their results, • check communication between components and • create reports about tests. User Groups CrashIt was designed for three different user groups that consequently demand dif- ferent requirements: Framework-developers write modules for the framework. They know its core functionality and are able to extend it or integrate the framework into other software development-tools. 2 Testing is defined as a process with the aim of finding differences between the implementation and the specified requirements. The premise is that the specification is correct.
  • CHAPTER 6. CRASHIT - A SHORT INTRODUCTION 58 Component-developers write components to be tested by CrashIt . Test-engineers start unit- or integration-tests over component(s). This user group is able to write complete test-configurations (see for more details section A.3 on page 96). Furthermore, test-engineers analyze test-results and try to give hints where bugs could be located, too. [Bec00] suggests that not only com- ponent-developers should test, but also customers, because they know the requirements of the product. 6.2 Main concept CrashIt can be used as a stand-alone application or as an ant-task [Apa03] (see 8.3 on page 83). The descriptive test data are stored in several files. CrashIt uses XML-files3 that include these necessary test-configurations. A test is separated into different parts and each part is defined in an individual file. Test-case : It models one method call. It contains • how the call has to be done by defining the TestClass, • what should be done by defining the method and all needed parameters and • how the result should be verified (ResultChecker). Test-sequence : It combines several test-cases to a sequence. These sequences in- clude flow-control information that can be used to skip tests if an unexpected result appears. The sequences define the test-cases and refer to a connection description that is used to set up a test scenario. First, this connection de- scription is be used to initialize and connect the needed components. After this has been finished the test-sequence will be executed. This is necessary to 3 The current version supports only xml files, but it is possible to define an own file-format or an other source that stores the configuration e.g. a database.
  • CHAPTER 6. CRASHIT - A SHORT INTRODUCTION 59 guarantee that each test-sequence uses new components that do not include information of older test runs. The connection description is spitted in to a definition of components and its relations. It can additionally include contract-components that can be used as proxies between two components 4 . Test-configuration : It defines a whole test scenario, including different test- sequences and configurations. The sequence of the defined test-sequences in- cludes also information for the flow-control, so that it is possible that some test-sequences are be skipped or - depending on some results - other test- sequences are be invoked. Defining this configuration in several separated files provides the possibility of only reading data needed for the test. Files including data that would be used by skipped test-cases or test-sequences are not read. If the test-configuration was stored in only one file, the whole file would have to be read before the test starts even when a lot of test-cases and test-sequences are skipped. 6.3 Testing topics covered by CrashIt The framework has been designed to support developers and test-engineers in test- ing applications and their components. As mentioned above there are different approaches to run tests on software components. Some of these approaches can be implemented by using CrashIt . Chapter 5 in [Val03] includes a detailed explanation of how CrashIt can be used to increase the quality of a software project by addressing the different approaches. The next sections will only give a quick overview of how this can be done. 4 These proxies are noted in 2 on page 52 and refer to the concept described in [Val03, Chapter 4,p. 37]
  • CHAPTER 6. CRASHIT - A SHORT INTRODUCTION 60 6.3.1 Correctness The correctness of a component can be verified by invoking methods of this compo- nent and comparing the results with expected values. This can be done in CrashIt by using the ResultChecker of the applied test-case. ResultCheckers are a more general approach of verifying the impact of a method-call than the assert-methods of JUnit. The ResultCheckers can for instance be used for verifying if a database entry or a file has been written (see 24, page 80 for details). It is easy to implement and integrate its own ResultChecker to verify special impacts of method calls which are not covered by existing ones. 6.3.2 Robustness The robustness of a program can be tested by applying method-calls with wrong parameters or in an wrong sequence. Even for checking this quality factor the ResultCheckers are useful, because they can verify if everything is still consistent despite the appearance of a failure. The provided reasons and hints, e.g. in a thrown exception, can also be verified by a ResultChecker. Testing for robustness is even as important as testing for correctness. 6.3.3 Compatibility It will be possible to use CrashIt in two ways for compatibility-testing. First, it is possible to implement proxies that verify and record if a component behaves as it has been defined in a general description for this component type. Interfaces and components that implement it can be tested by using this approach. It is also possible to do this in JUnit by implementing an abstract TestClass that uses only the interface definition of a component. An inherited TestCase that instantiates a concrete implementation for the interface can be implemented and this test can also verify if the component behaves as described in the design. These
  • CHAPTER 6. CRASHIT - A SHORT INTRODUCTION 61 abstract TestClasses can be implemented after the interfaces have been defined as specified in extreme programming (see 68). In this approach it is assumed that the two implementations are written in the same language. This leads to the second way of compatibility tests by using CrashIt that was one of the reasons for starting this project. This method is not yet available, but it is planned to realize it in the near future. CrashIt is now able to test Java components and these tests are defined in a language-independent way. If an implementation of CrashIt is available for other languages or other platforms, it is possible to use the same test-configuration for testing the components, because the test has been defined in XML-files and has not been implemented in a specific language.
  • CHAPTER 7. USING CRASHIT IN THE SOFTWARE DESIGN CYCLE 62 Chapter 7 Using CrashIt in the software design cycle There is no useful rule without an exception. Thomas Fuller This chapter will show how CrashIt can be used in different software development models. It will not provide a single way for all development models but it provide ideas how CrashIt can be used during the development cycle. Test-benches are traditionally closely bound to extreme programming but all other models also need test-benches to verify the development progress. A development-process consists of several topics that interact with each other. De- sign models are schemes how this abstract topics can be combined. The topics and its relations are part of a methodology.
  • CHAPTER 7. USING CRASHIT IN THE SOFTWARE DESIGN CYCLE 63 7.1 Methodology concepts A methodology is defined as a series of related methods or techniques for reaching a concrete goal. There are 13 (see figure 7.1) elements that define the elements of a methodology[Coc02, chapter 4,p.115]. Figure 7.1: Elements of a methodology These elements1 can be used in each team endeavor, whether it deals with software or not. When the defined steps of a development-model are generalized it can also been applied to none software projects. The testing environment is related to almost every methodology element. It is part of the used tools and techniques, it forces the team members to increase their skills, it helps to increase the quality of the product and will be necessary to verify if a certain milestone has been reached. Testing has also an influence on the required activities, sometimes new activities like implementing test classes and running the 1 A detailed discussion of the elements can be found in [Coc02, 115-120]
  • CHAPTER 7. USING CRASHIT IN THE SOFTWARE DESIGN CYCLE 64 tests are necessary and sometimes activities will become useless, e.g. a test-tool is able to generate the documentation of the test run. Testing is therefore one key topic in the development process. 7.2 Development-models There are different models that are based on different - sometimes historical - ap- proaches. Typically they focus on processes, milestones and activities, but all other methodology elements have a strong influence on the whole development process. The next sections will quickly repeater the concepts of the most famous models. A detailed description of the different models is not necessary for the rest of this chapter but it is useful to understand their main concepts. 7.2.1 The waterfall model This is one of the oldest models. It is based on the assumption that the development cycle can be separated into different independent phases. These phases are passed through one after the each other. The original model did not support a step back to a previous phase but the need of iteration has been quickly recognized. The second assumption that a phase is a strictly separated item has also been revised in newer models. A new phase can only start when the previous one has been finished. When a phase failed, new requirements must be defined and depending on the consequences one or more phases must be passed again. There are typically seven steps in the model: 1. Document the system concept. 2. Identify system requirements and analyze them. 3. Break the system into components (architectural design).
  • CHAPTER 7. USING CRASHIT IN THE SOFTWARE DESIGN CYCLE 65 4. Design each component (detailed design). 5. Code the system components and test them individually (coding, debugging, and unit testing). 6. Integrate the pieces and test the integrated components (integration test) 7. Test the whole system (system testing). 8. Deploy the system and operate it. Figure 7.2: The waterfall model The advantage of the waterfall-model is that the development process can be strictly planned, including milestones and deadlines, because no phase overlaps with a pre- vious one. The disadvantage it that there is nearly no time for reflection or revision and going back to the previous step is quite expensive.
  • CHAPTER 7. USING CRASHIT IN THE SOFTWARE DESIGN CYCLE 66 The model is useful for small and simple projects that use well known technologies as in such an environment projects can be easily and appropriately calculated. CrashIt and the waterfall model Planning the tests-scenario should be done as early as possible but in this model it is useful to start at that point, where the pieces of the system have been designed. At this point it is possible to define the proxies between the components that can be used for monitoring the communication between them. The next step is to define the tests for each component during its design. The idea of writing the tests before the code has been written is part of eXtreme Programming but fits very well even in this development model. If it is simple to define tests for the component it is in all probability even simple to use it. During the implementation of each component it can be tested by using the defined test-cases in the previous phase. The system- and integration-tests can be done by combining components by using the connection-configuration. As mentioned above the connection-configuration will be applied before a test sequence is invoked. 7.2.2 The spiral model The basic idea of the spiral model is an evolutionary development process that uses the waterfall model for each cycle. Analyzing the risk of each step is one main idea. Components with high priority will be designed and implemented first, reviewed, shown to the users so that the gained feedback can be used in the next cycle. Each cycle starts with a risk assessment in order to lead the next steps in the right direction. The spiral model is one of the currently used development models. Its concepts have big influence on other models or commercial solutions like the Rational Unified Process.
  • CHAPTER 7. USING CRASHIT IN THE SOFTWARE DESIGN CYCLE 67 Figure 7.3: The spiral model The advantage of the spiral model it that it focuses on re-use and error limitation. Generally it puts software quality factors up to front and tries to minimize the risk of the development. The feedback of users is an important part of each development process and is integrated into the spiral model. The disadvantages of the model are that it must be adapted for each project as it is not generally applicable and it causes overhead by the risk assessments. CrashIt and the spiral model CrashIt can be used during each cycle when the current implementation must be verified. According to its usage in the waterfall model it can be used in the related steps.
  • CHAPTER 7. USING CRASHIT IN THE SOFTWARE DESIGN CYCLE 68 7.2.3 Prototyping Prototyping is based on the idea to implement a basic prototype and enhance it step by step. It follows similar steps like the previous models but has its focus on re-designing and implementing a sample solution The advantage of prototyping is that in each phase a running solution is available that can be used for tests or evaluating the user’s acceptance. Its disadvantages are that the complexity of a grown program can quickly become unrulable and that it forces code-and-repair rather than a good design. CrashIt and prototyping CrashIt can be used to verify the prototypes after one step has been finished. This model makes the need of integration tests necessary. but it may cause big efforts to adapt the unit test between two circles, because the interfaces may strongly change during a re-design. The descriptive approach of defining the tests that is used in CrashIt may allow a faster update of the test-cases that implement a test for the new version. 7.2.4 Extreme programming Extreme programming is a combination of several successful ideas. It covers four values: communication, simplicity, feedback and courage and defines four basic ac- tivities: coding, designing, listening and testing. Other famous aspects of XP are: The Planning Game : It combines the technical estimates and business priorities to define the next milestone. Small releases : A simple system should run as quickly as possible. This running solution will be enhanced in small short release cycles.
  • CHAPTER 7. USING CRASHIT IN THE SOFTWARE DESIGN CYCLE 69 Refactorings : Changing the system structure without changing its behavior. This may force to reliability and performance of the system. Simple design : The system design has to be as simple as possible. Programmers should avoid solving problems that are parts of future releases. Pair programming : All code will be written by two programmers using one machine. It is more unlikely that two developers do not see a bug that one. Coding standards : All developers write their code in accordance with naming and design rules. This will simplify the communication between the involved developers. Testing : Programmer write test-cases to demonstrate that a feature has been implemented. The tests will be written before implementing a new feature. Extreme programming is an interesting mixture of techniques and concepts but it seems not to be applicable for all projects, because implementations of the tech- nologies or distributed applications had to be planned precisely. It can be hard to implement such applications from the scratch by redefining and enhancing a proto- type or it may cost big efforts for Refactorings. If a running prototype exists XP seems to be the right model especially when the customer is involved in design decisions. Figure 7.4 shows the life cycle in XP where the customer is able to define features that has to be included in the next cycle [JC00, chapter 2,p.13]. In well-established teams it is possible to define an amount of man-power that is used for refactorings and internal improvements. The remaining amount can be assigned by the costumer to implement important features. This provides a good balance of increasing the features and maintaining the code. Some aspects like writing the tests before starting to implement a feature might look academic, but writing a test is similar to using a component and the test is therefore the first application of this component and writing the test is the first verification of the design. Other aspects like pair-programming will contradict business guidelines.
  • CHAPTER 7. USING CRASHIT IN THE SOFTWARE DESIGN CYCLE 70 Figure 7.4: Life-cycle in XP Extreme programming is a controversial model as it defines extreme ways for solving problems. However, it is an interesting approach that can be adapted and used as a basis for successful developments. CrashIt and XP Testing is one central concept in XP so CrashIt can be used, as a test-bench in this approach to verify that a feature has been successfully implemented. Also, in this model the descriptive approach that is used by CrashIt to define the test-cases simplifies the changing of the test-case because the descriptive data are independent from the current implementation. A test can be run even when some interfaces or classes have not been fully adapted because no test-class must be compiled. In JUnit for example a test can only be run if all interfaces have been fully adapted. Otherwise it is not possible to compile the test-class.
  • CHAPTER 7. USING CRASHIT IN THE SOFTWARE DESIGN CYCLE 71 7.3 Using CrashIt in different models As mentioned above CrashIt can be used in each model as there must be a proof that a component has been correctly implemented. Not all models define the time when test-cases should be defined but a general rule is to write the test-cases as early as possible. 7.3.1 Which way is the best? There is no general rule when a model should be used. It depends on the application and the experience of the developers. The approach used by XP might be a bit extreme but forces developers to think very early about the tests. For implementing a new application XP might miss some important points of the application, but for maintaining a product in cooperation with users it might be the right way. The next steps are a proposal of combining the different models during the devel- opment cycle. Analyzing : Start with analyzing the user-requirements and derive the software- requirements from them. Design : After the software- and user-requirements have been written down the needed components and modules can be specified. This will result in an overall design including the main interfaces between the modules. Walk-throughs of the core functionalities are necessary to verify the design. During this phase it is useful to define the tests of the main components and integration tests. Defining the tests improves the information obtained during the walk-through because this information is written down in the test as valid usage of the components. The test scenario should also include invalid calls to verify if the components are able to notify a misusage. For this aspect thinking of the tests early in the design is necessary because otherwise it can be impossible to test the component as the it does not provide methods for verifying a misusage or
  • CHAPTER 7. USING CRASHIT IN THE SOFTWARE DESIGN CYCLE 72 does not throw exceptions. Implementing a prototype : After all interfaces and the tests have been written down the first prototype can be implemented. The prototype is finished when all test-cases are successful. Iterative development : Now new features and required refactorings can be dis- cussed on a running system and an iterative development can be started. This process iterates over several steps until the development has been finished: 1. Defining the necessary changes. 2. Alter the test-cases. 3. Implementing the changes. 4. Verifying the implementation if it fulfills the tests. These steps combine some concepts of all models in order to focus on testing the application. There are only few environments where software is necessary that must not include bugs e.g. applications in medical environments or applications in aircrafts. For almost every other application users have accepted that is not possible to implement it without a bug. As IT solutions are having more and more influence in our lives it becomes more important that software does not include bugs. However, it is a goal that it hard to reach but it must still be the goal of each developer. Focusing on testing during the development will force stable, reliable and application without or fewer bugs.
  • CHAPTER 8. THE IMPLEMENTATION OF CRASHIT 73 Chapter 8 The Implementation of CrashIt Nothing is particularly hard if you divide it into small jobs. Henry Ford The goal if this chapter is to explain the internal structure of CrashIt . This may help developers to extend the framework. It is not necessary to implement addi- tional components when CrashIt is used as test-bench but in some cases it makes sense to implement its own components. If someone wants to implements an addi- tional component for CrashIt please contact the development team, because it can be possible that a similar component is being planned or that the component will be part of a further release. 8.1 Overall architecture The overall architecture of CrashIt follows the main principle that it is to be be pos- sible to replace everything that is useful to exchange and to provide a simple way of integrating new components or extensions.
  • CHAPTER 8. THE IMPLEMENTATION OF CRASHIT 74 This requirements forces the framework to support a declarative configuration that can be widely adapted without recompiling components of the framework. It is possible to write its own configuration-subsystem but a full-featured XML-based version is part of the actual CrashIt implementation1 . Figure 8.1: CrashIt modules The architecture consists of several main parts: Configurator : This component is responsible for initializing several sub-components. It therefore is uses a configuration object including all information on the con- figuration. The configuration is able to load and connect the components that are necessary for the test. 1 For own versions implement org.dinopolis.crashit.configurator.config.Configu- ration and see org.dinopolis.crashit.impl.TestApplication how a configuration can be used
  • CHAPTER 8. THE IMPLEMENTATION OF CRASHIT 75 Test-engine : This component is responsible to run the tests. It uses a test- configuration that holds all information about the test. ContractAdministrator : This component is responsible for managing contracts which are used during a test. These contracts can interact with each other and they therefore use the contractadministrator which is able to return the available contracts for a test run. Logger : CrashIt provides a configurable logging interface that allows to use differ- ent logging mechanisms. An implementation of a logging framework - carried out by two students of Graz University of Technology - and an adapter for using Log4J as logging mechanism in CrashIt are included in the current dis- tribution. ResultSummary : This component is able to create summaries of what happened during the test. This is a simple overview of the components that are part of the whole framework. A detailed description of all components can be found in [Val03, chapter 6]. 8.2 Interfaces and design-decisions It is necessary to explain design-decisions and to understand its resulting interface- structure to use a framework as basis of an application or to extend it. The next sections point out these decisions and explain how the framework can be used to implement a test-application. 8.2.1 The CrashIt environment One real important class of the framework is org.dionopolis.crashit.CrashIt- Environment. This is an abstract class that implements several static methods for registering and creating instances of CrashIt -components. CrashIt -components
  • CHAPTER 8. THE IMPLEMENTATION OF CRASHIT 76 themselves must implement the interface org.dinopolis.crashit.CrashItFramework- Component that is used to register, initialize and shut down a component-instance. Each concrete implementation of a CrashIt -environment should be done by inherit- ing from this class, because the static methods of the abstract environment class are used by several other components within the framework. The CrashItEnvironment can be interpreted as a singleton [GHRV95] that is used in the whole framework. As it does not provide a static getInstance() method it is a pseudo-singleton2 . The singleton pattern is a very useful design pattern but it has one big drawback especially when it is used in Java. There is no simple way of generalizing a single- ton because the singleton pattern is based on the assumption that a static method getInstance returns the only instance of this class. As it is not possible to de- fine static methods in interfaces it is not possible to generalize a singleton in Java. However, it is necessary to use the CrashItEnvironment as a base class for fu- ture implementations. Consequently two further interfaces have been defined. The org.dinopolis.CrashItFramework interfaces provides methods for configuring the framework and org.dinopolis.FrameworkConfigurator) is used to set up concrete CrashItFramework implementations. These implementations have to be realized as singletons3 and they have to inherit the abstract CrashItEnvironment class. This seems to be a rather complicated approach to solve the problem, but when it is done in this way the pseudo-singleton class can be used in the whole framework for addressing the components and a general configuration of the framework is possible, too. 2 Pseudo-singleton means that the class behave like a singleton because this class is the only way for accessing the registered components but do not support the instantiation of a singleton object. 3 The package org.dinopolis.crashit.xml includes the implementations of this interfaces that are required for the XML implementation
  • CHAPTER 8. THE IMPLEMENTATION OF CRASHIT 77 Figure 8.2: UML diagram of the environment classes 8.2.2 Accessing the configuration, the configuration-layer As mentioned above the configuration of the framework can be done in a very flexible way. This is true for the used format, but internally there are some interfaces that are used to represent the configuration. The configuration-subsystem of CrashIt is responsible for converting the stored information into objects that can be accessed by these interfaces. There is one important pattern that has been used in the
  • CHAPTER 8. THE IMPLEMENTATION OF CRASHIT 78 design of these internal configuration interfaces, the Iterator - pattern[GHRV95]. All objects that hold data of the configuration are implemented as lists that support an iterator for accessing its elements. The elements of these lists are value-objects that store the required data (e.g. org.dinopolis.crashit.configurator.config.De- scriptionOfComponent). These structures allows the lists to read the configuration exactly in that moment when it is needed. If some data are not needed because their corresponding test-task has been skipped these data will not been read. Figure 8.3 on page 79 shows these internal interfaces. 8.2.3 Using the framework CrashIt is designed as a framework, but how can it be used for writing a test- application? There are two implementations that use CrashIt for realizing a test- bench (see 8.3 on page 83). An application that uses this framework for setting up a test application has to proceed the following steps4 : 1. Load a FrameworkConfigurator. This can be done by using a class-loader. 2. Creating a new CrashItFramework instance by invoking the createEnviron- ment() method of the configurator. 3. Applying the test-configuration to the CrashItFramework instance. 4. Initializing the framework by invoking the static method initialize of the CrashItEnvironment class. After these steps have been applied the framework can be used to run the configured test. The test is started by invoking the method run() of the CrashItEnvironment class. 4 See the source code of class org.dinopolis.crashit.impl.TestApplication for an imple- mentation of these steps.
  • CHAPTER 8. THE IMPLEMENTATION OF CRASHIT 79 Figure 8.3: UML diagram of internal configuration-interfaces After the tests have been finished the shutdown process has to be signaled by in- voking initializeShutDown and will be finished by calling the method shutDown. Splitting the shutdown process in two steps was done to provide a chance for the involved components to shut down on a concurrent way, so that there is enough time to close all opened files or database connections.
  • CHAPTER 8. THE IMPLEMENTATION OF CRASHIT 80 8.2.4 Interfaces for test-cases The last sections described some important interfaces for the configuration and its usage. Beyond these examples there are several other interfaces and components that are important especially for describing the test-case. It is even more possible for one of these components to be implemented for a special test-application than for a new configuration to be written. The test case is the smallest part of the test configuration. A test-case is slitted into two parts. TestClass : This part defines how the test will be applied. For example, a method of a local component is be executed or a method of a remote object is called. Separating the invocation of a method in an own class was done to increase the possibilities of compatibility-testing5 . A new type of invoking a method can be easily realized by implementing the interface org.dinopolis.crashit.testengine.TestClass and can be integrated in the test by defining it in the test-configuration. ResultChecker : The ResultCheckers are a more general approach of verifying the result of a method-call. Verifying the result of a method-call is the main- concept of running a test. Other test-frameworks solve this problem by pro- viding several methods that compare the result of the call with the expected result. But what if the call does not return a result because it uses a void method, or how to verify that it is true that a database entry has been writ- ten by the method? In these cases the result can only be checked by either invoking a further method or by implementing additional functionality that verifies the result. Both solutions do not exclude that an error was hidden by the additional code that is needed to verify the result. ResultCheckers can be used to implement this additional code only once. They 5 If other components are be loaded, it is necessary to re-implement or extend the current component factory which is responsible for creating the objects that are to be tested.
  • CHAPTER 8. THE IMPLEMENTATION OF CRASHIT 81 will be verified and test-engineers can trust them. For each object-type that is to be verified in a test-run an appropriate ResultChecker must exist. This seems to cause an additional effort for the test-engineer, but if a checker has been written it can be used in other test-cases, too6 . ResultCheckers can be realized by writing a class that implements the org.di- nopolis.crashit.testengine.ResultChecker interface and that can be eas- ily used in a test-run as it is enough to define them in the test-configuration. 8.2.5 The XML-subsystem As mentioned in the last sections the configuration is one of those topics that has the biggest influence on the framework-design. CrashIt supports a configuration-layer and provides a full-featured XML-based implementation of these interfaces. The XML-based implementation itself uses a small framework for defining and extending XML-configurations. This XML-framework is part of CrashIt and is based on the concept of tag-libraries. A SAX-based parser reads the XML-file that stores the configuration and depending on the type of the file it uses different tag-configurations for processing the configuration. The reason for this approach is the fact that SAX-based parsers will become quickly unmanageable and in some cases it is therefore rather complex to extend their func- tionality. If a system that uses this tag-library-based approach has to be extended, the mixture of describing the new tags and implementing their additional function- ality in seperated classes provides a controllable development because the relations and semantics of the tags are separated from their functionality. The tag-library itself uses an XML-format for storing the tag-names, their corre- sponding classes and the relations between the different tags. 6 The current implementation of CrashIt supports several basic object-type ResultCheckers but in the future the number of available checkers is likely to increase.
  • CHAPTER 8. THE IMPLEMENTATION OF CRASHIT 82 1 <t a g l i b > 2 <name>C o n n e c t i o n L i s t Tags</name> 3 <owner>o r g . d i n o p o l i s . c r a s h i t . xml . t a g s . c o n n e c t i o n s . C o n n e c t i o n Li s t O w n e r</owner> 4 <t a g s > 5 <tag> 6 <name>c o n n e c t i o n l i s t </name> 7 <c l a s s >o r g . d i n o p o l i s . c r a s h i t . xml . t a g s . c o n n e c t i o n s . C o n n e c t i o n s L i s t T a g</ c l a s s > 8 <a t t r i b u t e s > 9 <a t t r i b u t e > 10 <name>path</name> 11 </ a t t r i b u t e > 12 </ a t t r i b u t e s > 13 <n e s t e d t a g s > 14 <tag>c o n n e c t i o n</tag> 15 </ n e s t e d t a g s > 16 </tag> 17 <tag> 18 <name>c o n n e c t i o n</name> 19 <c l a s s >o r g . d i n o p o l i s . c r a s h i t . xml . t a g s . c o n n e c t i o n s . C o n n e c t i o n L i s t T a g</ c l a s s > 20 ... Listing 8.1: CrashIt tag-library example Each tag-library definition is parsed and mapped into an internal representation a org.dinopolis.crashit.xml.TagContext. This context can be used to generating instances of the tags which have been defined in the tag-library file. This context is used by a parser to process concrete configuration files. The parser also needs a concrete handler that stores the processed information. This is handler is called TagContextOwner. Figure 8.4 shows an incomplete UML diagram of the configuration tags. The StartTag extends the ConfigBasicTag wich extends itself the BasicTag. The ConfigBasicTag holds a reference on the concrete TagContextOwner for this con- figuration environment the ConfigurationOwner. Parsing and generating the tag-context is an expensive task and should only be done once for each file. A manager (org.dinopolis.crashit.xml.TagContext- Manager) has been therefore implemented that can be used to generate a context. This manager caches generated tag-contexts to optimize the process of reading the configuration files. A tag-library-based approach is a common way of handling complex configurations or applications. It can be found in popular projects like JavaServerPages, J2EE, or ant.
  • CHAPTER 8. THE IMPLEMENTATION OF CRASHIT 83 Figure 8.4: UML diagram of the configuration tags (incomplete) 8.3 Available applications As mentioned above CrashIt is designed as a framework but there are also two applications available that are based on it, a stand-alone application (org.dinop- olis.crashit.impl.TestApplication) and an ant-task (org.dinopolis.crash- it.ant.CrashItAntTask. Both implementations uses the XML-configuration, the same configuration can therefore be used in both. Notice that depending on the logging sub-system additional libraries has to be visible in the classpath. CrashIt uses Log4J as default logging framework.
  • CHAPTER 8. THE IMPLEMENTATION OF CRASHIT 84 8.3.1 Stand-alone Application This application is implemented as a Java-Class that can be started by: java org.dinopolis.crashit.ant.TestApplication <config-file> The <config-file> refers to a file that stores the main configuration for the test. A detailed description of this file can be found in section A.3.1 on page 97. 8.3.2 Ant-task Adding an ant-task to a build.xml is simple as ant provides a method of integrating new tasks. The new task must be visible in the classpath of the environment where ant will be used. The antCrashIt -task can be used by adding the following lines to the build.xml file: 1 <t a r g e t name=” t e s t ”> 2 <t a s k d e f name=” c r a s h i t ” 3 c l a s s n a m e=” o r g . d i n o p o l i s . c r a s h i t . ant . C r a s h I t T a s k ”/> 4 < c r a s h i t t e s t f i l e =”<c o n f i g −f i l e >”/> 5 </ t a r g e t > Listing 8.2: CrashIt as ant-task When the target test will be invoked by calling ”ant test”CrashIt will start. In- tegrating CrashIt into an existing development-environment that is based on ant is therefore very simple.
  • CHAPTER 9. CONCLUSION 85 Chapter 9 Conclusion 9.1 Personal experience Designing and implementing CrashIt and the need of dealing with topics like XP (eXtreme Programming) surely increased my faculties in designing software. The work-flow defined in chapter 7.3.1 will be the way of developing future projects and this work will therefore have a strong influence in organizing and developing future projects. The teamwork and the focus on designing and planning before staring to implement made it possible to comply with all milestones and pointed out that this is one successful methodology in software-development. It was an interesting and exciting year that I do not want to miss. 9.2 Project related topics The actual version of CrashIt can be seen as the core of a bigger framework that will be implemented in future. It will now be used in a lecture about software design and thereby hopefully a lot of feedback can be treated in future releases. This version is the successful prove that the ideas of CrashIt can be used for testing
  • CHAPTER 9. CONCLUSION 86 component based systems. CrashIt has the some disadvantages in comparison to existing test-benches. • Now a all configuration-files must be written by hand. This will be improved when an integrated environment is available. This environment will emphases the advantages of CrashIt • Missing integration into any existing IDE. Integration can now only be done by using the ant-task ([Apa03]) The biggest advantage of CrashIt is up to now is that no test-code must be written and therefore the test-team van start to design and implement the test parallel to the development team without the need of existing interfaces. An other advantage of CrashIt is its modular design so that it should be easy to include it into an other environment e.g. testing J2EE applications. CrashIt is now available on crashit.hti.at but will in future be available on www.dinopolis.org 9.3 Outlook There are several projects that are pending or are in progress. Result Store the result-store of CrashIt has been redesigned by a group of students and will be included into CrashIt during this term. C++ porting of CrashIt or reimplementing it as a test bench for C++ programs will be part of future master thesis or dissertations. J2EE during a pending J2EE project the implementation of an J2EE version of CrashIt is planned.
  • CHAPTER 9. CONCLUSION 87 automated test-sequence generation this will be part of a following disserta- tion. Some theoretical work on this topic has been done during the design of CrashIt but this must be reviewed and tested before it will be publicized.
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 88 Appendix A The usage of CrashIt - a simple example A.1 Introduction The intention of this example is to give a short introduction in designing component based frameworks and to show the usage of CrashIt on a small component based ex- ample. In this example serveral components define a small framework for managing string based services. This example uses the hook Hook pattern. The hook pattern is one of the basic patterns. For understanding the concepts of CrashIt and its usage in this simple example it is sufficient to know about the hook-pattern and its application. Now lets have a look on this concret example of a small component based framework. A.2 A service example Sometimes it is necessary to administrate different services. For example, storing a document on a local hard disk or storing it on a FTP (File Transfer Protocol)
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 89 server can be described by using similar parameters. In the first version of an application storing documents on a FTP server may not be supported. By applying an update, this additional functionality can be added easily if the mechanism for storing a document is clear separated and, for example, implemented as a service of the application. In this example a service is simpler than storing a document, but the example uses a similar mechanism for implementing services. In this example a service is able to deal with strings and to perform calculations on these strings. Consequently, a method String execute(String) is therefore used that processes the given string value. A.2.1 Parts of the framework The framework consists of three different parts: Service: It processes a string value and returns the result as another string. A Service must be initialized and throws an exception if it is used before it has been initialized. It is possible to check if the service has been initialized or not. A Service hast to provide an id through which a provider may address it. ServiceProvider: It manages different services. The ServiceProvider1 can ini- tialize all registered services. A service can be used for processing a certain value by passing two parameters to the ServiceProvider, the id of the service that is to be used and the value that is to be processed. Configurator: It can be used to configurate the ServiceProvider by applying a concrete Configurator to the ServiceProvider.2 The Configurator regis- ters the different Services and forces the provider to start their initialization. 1 The service provider can be interpreted as a Mediator [GHRV95, Cap. 5]. 2 The Configurator used in this example can be configured by a property file. This is a simple method to keep the configuration changeable.
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 90 The implementation of this framework is achieved by defining interfaces for each part and using these interfaces to implement concrete components. For minimizing the effort of component development an implementation of an abstract class should be done. If a new component is needed, the programmer must only extend the abstract class, so the benefits of inheritance and using interfaces can be combined. A.2.2 Interfaces There are three interfaces that correspond to the three different parts of this example framework: Service Interface The Service interface (see listing A.1) consists of four methods: • getServiceId • execute • init • isInitilized This is a very small interface because an implementation that can be used in a real world environment would be too extensive for this example. A service interface in an real world implementation should possibly contain mechanisms for supporting differ- ent versions, compatibility checks, a mechanism for applying additional parameters or the chance to initialize a roll-back. ServiceProvider Interface The ServiceProvider interface (see listing A.2) is very small too. Even a smarter implementation of a provider would go beyond the scope of this example. The
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 91 1 package o r g . s e r v i c e s ; 2 3 // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 4 /* * 5 * This is a interface for a simple string - based Service . 6 * 7 * @pre : Before sending requests to a Service , which implements 8 * this interface , it should be initialized by calling 9 * { @link # execute ( String )}. 10 * 11 * @author Gerhard Fliess 12 */ 13 public i n t e r f a c e S e r v i c e 14 { 15 // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 16 /* * 17 * Method : getServiceId 18 * @return String 19 * 20 * Usage : This method returns the <b > id </ b > of the 21 * Service 22 * 23 * @package : org . services 24 */ 25 public S t r i n g g e t S e r v i c e I d ( ) ; 26 27 // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 28 /* * 29 * Method : execute 30 * @param value 31 * @return String 32 * @throws ServiceFailure : This exception would be 33 * thrown when the Service is unable to answer the 34 * request or when it is not initilized . 35 * 36 * Usage : The Service processes <b > value </ b > and 37 * returns the modified string 38 * 39 * @pre : isInitialized ()== true 40 * 41 * @package : org . services 42 */ 43 public S t r i n g e x e c u t e ( S t r i n g v a l u e ) throws S e r v i c e F a i l u r e ; 44 45 // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 46 /* * 47 * Method : init 48 * @throws ServiceFailure 49 * 50 * Usage : Over this method , the Service could be 51 * initialized . 52 * 53 * @pre : isInitialized () != true 54 * @post : isInitialized () == true 55 * 56 * @package : org . services 57 */ 58 public void i n i t ( ) throws S e r v i c e F a i l u r e ; 59 60 // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 61 /* * 62 * Method : isInitialized 63 * @return boolean 64 * 65 * Usage : This method returns <b > TRUE </ b > , if 66 * the Service is initialized , otherwise <b > FALSE </ b >. 67 * 68 * @package : org . services 69 */ 70 public boolean i s I n i t i a l i z e d ( ) ; 71 } Listing A.1: Service interface
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 92 1 package o r g . s e r v i c e s ; 2 3 // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 4 /* * 5 * A S ervic eProvider manages different string - based Services . 6 * All registered Services must implement the { @link Service } interface . < br > 7 * A S ervic eProvider can be initialized by registering all services or 8 * over a { @link ProviderC onfig } 9 * 10 * @author Gerhard Fliess 11 */ 12 public i n t e r f a c e S e r v i c e P r o v i d e r 13 { 14 // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 15 /* * 16 * Method : registerService 17 * @param service 18 * @throws ServiceFailure 19 * 20 * Usage : Over this method , Services can be registered . 21 * After all Services has been registered { @link # initServices ()} 22 * should be called . 23 * 24 * @pre : service . isInitialized () == false 25 * 26 * @package : org . services 27 */ 28 public void r e g i s t e r S e r v i c e ( S e r v i c e s e r v i c e ) throws S e r v i c e F a i l u r e ; 29 30 // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 31 /* * 32 * Method : initServices 33 * @throws ServiceFailure 34 * 35 * Usage : This method would invoke the 36 * initialisation of all registered Services . 37 * 38 * @post : For all service . isInitialized () == true 39 * 40 * @package : org . services 41 */ 42 public void i n i t S e r v i c e s ( ) throws S e r v i c e F a i l u r e ; 43 44 // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 45 /* * 46 * Method : ask 47 * @param serviceId 48 * @param value 49 * @return String 50 * @throws ServiceFailure : would be thrown if the 51 * Service is not available or another failure appears 52 * during the request . 53 * 54 * Usage : Over this method , a external user can 55 * invoke a the Service ( specified by a <b > serviceId </ b >) 56 * to process <b > value <b >. The answer of the 57 * Service would be forwarded to the external user . 58 * 59 * @package : org . services 60 */ 61 public S t r i n g a s k ( S t r i n g s e r v i c e I d , S t r i n g v a l u e ) throws S e r v i c e F a i l u r e ; 62 63 // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 64 /* * 65 * Method : config 66 * @param config 67 * @throws ServiceFailure 68 * 69 * Usage : This method is used to configurate the 70 * provider . The configuration is contained in 71 * { @link org . service . Provider Co nf ig }. 72 * 73 * 74 * @pre : config must be valid 75 * @package : org . services 76 */ 77 public void c o n f i g ( P r o v i d e r C o n f i g c o n f i g ) throws S e r v i c e F a i l u r e ; 78 } Listing A.2: ServiceProvider Interface
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 93 interface consists of four methods: • registerService • initServices • ask • config Corresponding to the extensions of the Service interface the ServiceProvider interface for a real world application should contain more methods than this simple one. Possible extensions for a provider could contain mechanisms for searching a service or at least a compatible service. Other possible extensions would be methods for registering initialized services and mechanisms to unregister services or versions of services. In order to keep it simple all important extensions are dropped. Configurator Interface 1 package o r g . s e r v i c e s ; 2 3 /* * 4 * Interface of a Configurator to initialize a 5 * { @link org . service . S erviceProvider }. 6 * 7 * @author Gerhard Fliess 8 */ 9 public i n t e r f a c e P r o v i d e r C o n f i g 10 { 11 // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 12 /* * 13 * Method : configure 14 * @param provider 15 * @throws ServiceFailure 16 * 17 * Usage : This method configures a ServiceProv i de r 18 * denoted by <b > provider </ b > 19 * 20 * @pre : provider must exist 21 * @package : org . services 22 */ 23 public void c o n f i g u r e ( S e r v i c e P r o v i d e r p r o v i d e r ) throws S e r v i c e F a i l u r e ; 24 } Listing A.3: Configurator Even the Configurator interface is very small to keep the example simple. In real world applications designers should avoid interfaces with only one or two methods.
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 94 If an interface is that, small the probability of a design failure or forgotten function- ality is relatively high. The Configurator interface of this example consists of only one method for config- uring it ServiceProvider. A configurator of a real world implementation should be more complex. For example, it should be possible to add additional parameters like the used configuratin file or there should be methods to define constrains for the initialized services. A.2.3 Implementations The version of this example supports two services: Echo : returns the given value Concat : concatenates the given values There are also implementations of a ServiceProvider (org.service.ServiceProviderImpl) and a configurator (org.service.SimpleProviderConfig). The configurator automati- cally registers services that are defined in the services.properties file. Next, two examples are given, which show how the framework can be used. The first one (see listing A.4) shows how the framework can be used without an configurator. These steps are important for defining test sequences for the framework. 1. an instance of the service provider must be created 2. both services are created and registered at the provider 3. the provider is forced to initialize the registered services 4. the services can be used by applying the corresponding key and a value
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 95 1 2 package o r g . s e r v i c e s ; 3 4 /* * 5 * This example shows how a provider can be initialized without 6 * a configurator . 7 * 8 * @author Gerhard Fliess 9 */ 10 public c l a s s S e r v i c e E x a m p l e 1 11 { 12 13 public s t a t i c void main ( S t r i n g [ ] a r g s ) 14 { 15 16 S e r v i c e P r o v i d e r p r o v i d e r = new S e r v i c e P r o v i d e r I m p l ( ” MainProvider ” ) ; 17 18 try 19 { 20 p r o v i d e r . r e g i s t e r S e r v i c e (new Echo ( ) ) ; 21 p r o v i d e r . r e g i s t e r S e r v i c e (new Concat ( ) ) ; 22 23 provider . i n i t S e r v i c e s ( ) ; 24 25 System . out . p r i n t l n ( ” Echo r e q u e s t : ” + p r o v i d e r . a s k ( ” echo ” , ” 12345 ” ) ) ; 26 System . out . p r i n t l n ( ” Concat r e q u e s t : ” + p r o v i d e r . a s k ( ” c o n c a t ” , ” 12345 ” ) ) ; 27 System . out . p r i n t l n ( ” Concat r e q u e s t : ” + p r o v i d e r . a s k ( ” c o n c a t ” , ” 12345 ” ) ) ; 28 29 } catch ( S e r v i c e F a i l u r e e ) 30 { 31 e . printStackTrace ( ) ; 32 } 33 34 } 35 36 } Listing A.4: A sample application without a Configurator The second one (see listing A.5) shows how a provider can be configured by using the Configurator. This version of setting up a framework follows the object-orientated approach and should be forced in designing frameworks3 : 1. an instance of the service provider must be created 2. the provider is configured by using the Configurator 3. the services can be used by applying the corresponding key and a value A test of this example would contain Testcase-sequences that test these two different setup methods and it would test the each used components separately. 3 A configurator follows the Delegator -pattern, see [GHRV95]
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 96 1 package o r g . s e r v i c e s ; 2 3 /* * 4 * This example shows the usage of a configurator . 5 * 6 * @author Gerhard Fliess 7 */ 8 public c l a s s S e r v i c e E x a m p l e 2 9 { 10 11 public s t a t i c void main ( S t r i n g [ ] a r g s ) 12 { 13 14 S e r v i c e P r o v i d e r p r o v i d e r = new S e r v i c e P r o v i d e r I m p l ( ” MainProvider ” ) ; 15 16 try 17 { 18 p r o v i d e r . c o n f i g (new S i m p l e P r o v i d e r C o n f i g ( ) ) ; 19 20 System . out . p r i n t l n ( ”Echo r e q u e s t : ” + p r o v i d e r . a s k ( ” echo ” , ” 12345 ” ) ) ; 21 System . out . p r i n t l n ( ” Concat r e q u e s t : ” + p r o v i d e r . a s k ( ” c o n c a t ” , ” 12345 ” ) ) ; 22 System . out . p r i n t l n ( ” Concat r e q u e s t : ” + p r o v i d e r . a s k ( ” c o n c a t ” , ” 12345 ” ) ) ; 23 24 } catch ( S e r v i c e F a i l u r e e ) 25 { 26 e . printStackTrace ( ) ; 27 } 28 29 } 30 } Listing A.5: A sample application using a Configurator A.3 The CrashIt Configuration CrashIt has an extensible design that allows developers to define their own file for- mats. The default format of configuration files used in CrashIt is XML (eXtensible Markup Language). (Even XML files are extensible, so CrashIt provides a frame- work to write new tags for the XML files.) For using CrashIt it is not necessary to write new tags but if new tags or additional properties are needed, the extension of CrashIt can be done easily. However, in such cases please contact the development team because it is possible that a similar extension is planned for the next release. The team can also provide help for developing extensions and it is possible that these extensions become part of the whole framework. The definition of a whole configuration is split into more files which can be stored in different directories. This split is justified by: • Testcase-configuration could be quite extensive or some of its parts may not be
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 97 processed, e.g. if previous tests failed (see also section A.3.8 on page 112). So it is futile to parse parts of the configuration which never would be performed. • It is possible, that some testcases or Testcase-sequences are executed several times. As a consequence it is futile to read files twice. CrashIt provides a cache which stores such files. In the whole configuration, files are referred by a unique id. There This is why, that there must exist further lists which map ids to the according files. Most con- figurations are separated into two types of files. One file defines mappings between a unique id and a part of the framework which is defined in a separate file. The files that contain the mapping are parsed during the setup of CrashIt, all other files will be parsed when they are needed. In the following sections the different configuration files will be described. There are several files that are needed to define a test which seems to be a big effort, but nearly all files can be written during the design and must be written only once. A.3.1 The Configuration file This is the main configuration file for a test run. It provides information where the Test-configuration can be found and additional information for the CrashIt en- vironment. The file is separated into two different parts that are embedded into a <configuration> ... </configuration> environment: • the definition of the Test-configuration <test>. . .</test> (see listing A.6 line 2-10). (For a detailed explanation, read section Test-configuration Related Part on page 98) • and the definition of the ResultStore <resultstoreconfig>. . .</resultstore- config> (See listing A.6 line 11-14). (For a detailed explanation, see the section ResultStore Related Part on page 99)
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 98 1 <c o n f i g u r a t i o n > 2 <t e s t > 3 <d i r >e x a m p l e s e r v i c e </ d i r > 4 <c o n f i g > 5 <d i r >t e s t c o n f i g u r a t i o n </ d i r > 6 < f i l e >t e s t c o n f i g u r a t i o n . xml</ f i l e > 7 </ c o n f i g > 8 <components>components . xml</components> 9 <c o n n e c t i o n s >c o n n e c t i o n s . xml</ c o n n e c t i o n s > 10 <c o n t r a c t s >c o n t r a c t s . xml</ c o n t r a c t s > 11 <r e s u l t c h e c k e r >r e s u l t c h e c k e r . xml</ r e s u l t c h e c k e r > 12 < t e s t c l a s s e s >t e s t c l a s s e s . xml</ t e s t c l a s s e s > 13 </ t e s t > 14 <r e s u l t s t o r e c o n f i g > 15 <c l a s s >o r g . d i n o p o l i s . c r a s h i t . r e s u l t s t o r e . impl . XMLResultStoreConfig</ c l a s s > 16 <r e s u l t >r e s u l t . xml</ r e s u l t > 17 </ r e s u l t s t o r e c o n f i g > 18 </ c o n f i g u r a t i o n > Listing A.6: The main configuration file Test-configuration Related Part Tags for the description of the Test-configuration are: <dir> (listing A.6 line 3): This is the root directory, where all Test-configura- tion files are placed. All subdirectories of <dir> are defined relatively to this directory. <config> (listing A.6 line 4-7): This tag is a frame for additional properties of the Test-configuration: • <dir>(listing A.6 line 5): Defines the subdirectory of the previously declared <dir> where the Test-configuration files can be found. • <file>(listing A.6 line 6): Defines the name of the file that con- tains the primary Test-configuration file (see also section A.3.2 on page 100) <components> (listing A.6 line 8): This tag defines the file which contains the mappings between the component ids and the XML files that describe these components. (Detailed informations about this file can be found in section section A.3.4 on page 106). <connections> (listing A.6 line 9): This tag defines the file which contains
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 99 the mappings between the connection ids and the XML files that define the connections (see section A.3.5 on page 107). <contracts> (listing A.6 line 10): This tag defines the file which contains the mappings between the contract ids and the XML files that specify the contracts (refer tosection A.3.6 on page 111) . <resultchecker> (listing A.6 line 11): Resultchecker are used to determine if a result is valid or not. This tag defines the file which contains the mappings between the Resultchecker ids and the XML files that define the properties of these classes. (Detailed informations about this mapping file is placed in 30 on page 105). <testclasses> (listing A.6 line 12): Testclasses are used to perform a single testcase on a component. This tag defines the file, describing the mappings between the Testclass ids and the appropriate XML file ResultStore Related Part This section describes the tags of a CrashIt -configuration which are used for the ResultStore. The development of ResultStore is not finished yet4 . This section describes the configuration of the current version. <class> (listing A.6 line 15): This tag defines the class which configures the ResultStore. A default class is actually given. <result> (listing A.6 line 16): Defines the filename into which the Result- Store stores the test results. 4 The implementation which is used in this version of CrashIt will be replaced by another one written by Arne Tauber and Mario Ivkovic.
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 100 A.3.2 The Test-configuration File The configuration of the Testcase-sequences consists of a file that merges the Test- case-sequence files into one configuration. A Test-configuration can be controlled by the results of the Testcase-sequences. This provides a Flowcontrol mechanism which allows to skip tests if something has gone wrong. Listing A.7 on page 101 shows an example of a configuration that does not use Flowcontrol. The tags in this file are even valid for configurations that supports Flowcontrol. <flow-control> (listing A.7 line 2): This tag is used to enable or disable the Flowcontrol. It supports the values on and off. <path> (listing A.7 line 3): This defines the directory that contains all con- figuration files for Testcase-sequences. It is relative to the directory that is defined in the configuration file (listing A.6 line 3). <testsequences> (listing A.7 line 4-11): This section defines the Testcase- -sequences and its order that should be applied during the test. Each Test- case-sequence is defined in a separate file. It is possible to define a variable, which stores the result of a Testcase-sequence. This is optional, but necessary if Flowcontrol is enabled, because Flowcontrol needs the results of the Test- case-sequences to decide which sequence is applied next. If Flowcontrol is not used, this section should only consist of test-sequence-tags. Listing A.8 on page 101 shows a configuration that uses the Flowcontrol mecha- nism. This file is not part of the service example. It is used to explain the syntax of a Flowcontrolled configuration. It is a simple configuration that uses only the if-construct. The if-section of the configuration must include a condition-section and a then-section. The else-section is optional. A condition can consist of a sim- ple compare statement or a boolean operation. Listing A.8 defines an if-statement
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 101 1 <t e s t c o n f i g u r a t i o n > 2 <f l o w −c o n t r o l >o f f </f l o w −c o n t r o l > 3 <path>t e s t −s e q u e n c e s</path> 4 <t e s t s e q u e n c e s > 5 <t e s t s e q u e n c e r e s u l t=” echo ”>t e s t E c h o . xml</ t e s t s e q u e n c e > 6 <t e s t s e q u e n c e r e s u l t=” c o n c a t ”>t e s t C o n c a t . xml</ t e s t s e q u e n c e > 7 <t e s t s e q u e n c e r e s u l t=” s e r v i c e c o n f i g t e s t ”>t e s t S e r v i c e C o n f i g . xml</ t e s t s e q u e n c e > 8 <t e s t s e q u e n c e r e s u l t=” echoFlow ”>t e s t E c h o F l o w . xml</ t e s t s e q u e n c e > 9 <t e s t s e q u e n c e r e s u l t=” c o n t r a c t s ”>t e s t C o n t r a c t s . xml</ t e s t s e q u e n c e > 10 <t e s t s e q u e n c e r e s u l t=” s e r v i c e t e s t ”> t e s t S e r v i c e . xml</ t e s t s e q u e n c e > 11 </ t e s t s e q u e n c e s > 12 </ t e s t c o n f i g u r a t i o n > Listing A.7: A simple configuration without Flowcontrol 1 <t e s t c o n f i g u r a t i o n > 2 <f l o w −c o n t r o l >on</f l o w −c o n t r o l > 3 <path>t e s t −s e q u e n c e s</path> 4 <t e s t s e q u e n c e s > 5 <t e s t s e q u e n c e r e s u l t=” s i m p l e ”>s i m p l e . xml</ t e s t s e q u e n c e > 6 <t e s t s e q u e n c e r e s u l t=” f i r s t ”> f i r s t . xml</ t e s t s e q u e n c e > 7 <i f > 8 <c o n d i t i o n > 9 <or> 10 <compare o p e r a t i o n=” e q u a l ”> 11 <t a r g e t >s i m p l e</ t a r g e t > 12 <v a l u e t y p e=” Boolean ”>t r u e</v a l u e > 13 </compare> 14 <compare o p e r a t i o n=” e q u a l ”> 15 <t a r g e t > f i r s t </ t a r g e t > 16 <v a l u e t y p e=” Boolean ”>t r u e</v a l u e > 17 </compare> 18 </or> 19 </ c o n d i t i o n > 20 <then> 21 <t e s t s e q u e n c e >s i m p l e . xml</ t e s t s e q u e n c e > 22 </then> 23 <e l s e > 24 <t e s t s e q u e n c e > f i r s t . xml</ t e s t s e q u e n c e > 25 </ e l s e > 26 </ i f > 27 <t e s t s e q u e n c e >b e a n t e s t . xml</ t e s t s e q u e n c e > 28 </ t e s t s e q u e n c e s > 29 </ t e s t c o n f i g u r a t i o n > Listing A.8: A Flowcontrolled configuration
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 102 (line 7-26) which consists of two compare operations which are combined by an or- statement. This statement written in pseudo code would look like this: 1 i f ( s i m p l e == true ) or ( f i r s t == true ) 2 then { apply ( simple . xml ) } 3 e l s e { apply ( first . xml ) } The names of the Testcase-sequence results simple and first can be interpreted as variables that store the result. Conditions can be more complex than in this example, as it is possible to combine boolean operations on results. Writing such a file causes more effort than writing a simple file, because the syntax of the Flowcontrol must be described in XML, too. Future versions of CrashIt will contain a graphical tool to define this sequences. Independent of the usage of the Flowcontrol mechanism the applied Testcase-sequen- ces are defined in separated files. These files can also define a plain Testcase-sequence or use the Flowcontrol mechanism. Testcase-sequence files refer to testcase files that are described in section A.3.3 on page 104. The definition of a Testcase-sequence consists of several parameters: <start-node> (listing A.9 line 2): This tag is used to define which component should be tested. The component is referred by its id (see section A.3.4). All test cases are applied on this component. <path> (listing A.9 line 3): This defines the directory, which contains the test- case descriptions for this Testcase-sequence. It is also a relative path. <flow-control> (listing A.9 line 4): This tag is used to enable or disable Flowcontrol. It supports two values on and off. <testcases> (listing A.9 line 5-10): This section contains the testcases that should be applied on the startnode or Flowcontrol tags. Each testcase is described in a separate file (see section A.3.3).
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 103 1 <t e s t s e q u e n c e > 2 <s t a r t −node>echo</ s t a r t −node> 3 <path>t e s t −echo</path> 4 <f l o w −c o n t r o l >o f f </f l o w −c o n t r o l > 5 <t e s t c a s e s > 6 <t e s t c a s e >g e t S e r v i c e I d . xml</ t e s t c a s e > 7 <t e s t c a s e >i n i t . xml</ t e s t c a s e > 8 <t e s t c a s e > i s I n i t i a l i z e d t r u e . xml</ t e s t c a s e > 9 <t e s t c a s e >e x e c u t e . xml</ t e s t c a s e > 10 </ t e s t c a s e s > 11 </ t e s t s e q u e n c e > Listing A.9: A simple Testcase-sequence without Flowcontrol 1 <t e s t s e q u e n c e > 2 <s t a r t −node>echo</ s t a r t −node> 3 <path>t e s t −echo</path> 4 <f l o w −c o n t r o l >on</f l o w −c o n t r o l > 5 <t e s t c a s e s > 6 <t e s t c a s e >g e t S e r v i c e I d . xml</ t e s t c a s e > 7 < t e s t c a s e r e s u l t=” i n i t i a l i z e d ”> i s I n i t i a l i z e d t r u e . xml</ t e s t c a s e > 8 <i f > 9 <c o n d i t i o n > 10 <compare o p e r a t i o n=” e q u a l ”> 11 <t a r g e t > i n i t i a l i z e d </ t a r g e t > 12 <v a l u e t y p e=” Boolean ”> f a l s e </v a l u e > 13 </compare> 14 </ c o n d i t i o n > 15 <then> 16 <t e s t c a s e >i n i t . xml</ t e s t c a s e > 17 <t e s t c a s e >e x e c u t e . xml</ t e s t c a s e > 18 <t e s t c a s e >g e t S e r v i c e I d . xml</ t e s t c a s e > 19 </then> 20 <e l s e > 21 <t e s t c a s e >e x e c u t e . xml</ t e s t c a s e > 22 </ e l s e > 23 </ i f > 24 </ t e s t c a s e s > 25 </ t e s t s e q u e n c e > Listing A.10: A Testcase-sequence using Flowcontrol
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 104 The Flowcontrol mechanism used for the Testcase-sequence is equal to the mecha- nism used for controlling the Test-configuration. Results of testcases can be named and used like variables, it is the same as the mechanism in the Test-configuration. Listing A.10 shows a Testcase-sequence that uses a Flowcontrol. A.3.3 The Testcase files These files defines a single testcase, which will be applied to some components by CrashIt . Listing A.11 shows a simple example of a testcase. Each testcase must be described in a own testcase file. This seems to be a big effort, but testcase files can be reused in other Testcase-sequences5 . 1 <t e s t c a s e > 2 <t e s t c l a s s > 3 <id >methodCall</id > 4 </ t e s t c l a s s > 5 <f u n c t i o n >e x e c u t e</ f u n c t i o n > 6 <r e s u l t > 7 <r e t u r n t y p e >S t r i n g </r e t u r n t y p e > 8 <e q u a l s >Otto</e q u a l s > 9 </ r e s u l t > 10 <p a r a m e t e r s > 11 <parameter> 12 <type>S t r i n g </type> 13 <v a l u e >Otto</v a l u e > 14 </parameter> 15 </p a r a m e t e r s > 16 <r e s u l t c h e c k e r > 17 <id >S t r i n g C h e c k e r</id > 18 </ r e s u l t c h e c k e r > 19 </ t e s t c a s e > Listing A.11: Calling a method Several parameters defines a testcase: <testclass> (listing A.11 line 2-4): A Testclass implements how a test is conducted. A unique id is applied to each Testclass. These ids are defined in a separate file. The testclass-tag uses the id to address the Testclass. So far only one Testclass has been implemented. This class is able to invoke 5 Future versions of CrashIt will contain a graphical tool for designing testcases and Testcase-se- quences.
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 105 methods of components that can be accessed locally. Future versions of CrashIt will include smarter Testclasses. For example, it could be useful to implement Testclasses that are able to invoke method calls via RMI (Remote Method Invocation). The definition of the used Testclasses is done in two simple files, a file that assigns the ids for the test classes, in this example the file testclasses.xml, and files for each test-class. From a more formal view the Test-configuration and the test-classes defines the method for the application of the test. <function> (listing A.11 line 5): This tag defines which method should be called on the component. <parameters> (listing A.11 line 10-15): This tag can contain several param- eter definitions. The parameters defined in this tag are used in the method call. Each parameter definition is embedded in a parameter tag that includes the subtagsthat are applied to the parameter converter described in section A.3.7: • <type> which defines the object-type of this parameter. • <value> which defines the value of this parameter Both will be preprocessed by a Parameter-converter, see also section A.3.7 on page 111. <result> (listing A.11 line 6-9): This tag defines which result is expected. Section A.3.7 describes the implementation of the general converter which is used for creating objects. The tag contains two sub tags: • <returntype> This defines the expected object-type of the return value. • <equals> This defines which value is expected. <resultchecker> (listing A.11 line 16-18): Resultcheckers are a more ab- stract approach of comparing the results of a testcase and the definition in
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 106 the specification. Simple Resultcheckers compare objects like Strings with the expected objects defined in the specification, however, a more complex result-checker can also be used to check a result without comparing it with another object. Resultchecker is also used for checking complex results, e.g. if a database entry was written or a file was created during the actual testcase. This feature also allows to check the result of a method that has a void return type, but is responsible to change the state of other components or resources. This can be done by using a result-checker that is able to monitor the related components or resources. Resultcheckers are defined in several files, too: one file that assigns the ids, (in this example resultchecker.xml) and another for each Resultchecker. A.3.4 The Component Files Components files define which component should be loaded and describe how this component must be initialized. They consist of a some tags, that are embedded in a component-tag: <class> (listing A.12 line 2): This tag defines which class is a component. The class mus be available in the classpath. If this class does not support a simple constructor, a section defining the needed constructor parameters must be added into this file. <constructor> (listing A.12 line 3-8): This section is optional if a compo- nent supports a simple constructor. However, it is necessary if no simple default constructor is defined. It consists of one ore more parameter sections. These parameter sections are equal to the sections in the testcase file (see section A.3.3 or listing A.11 line 10-15 ) (A detailed description can be found in section A.3.7). If a constructor is described, it will be called after the component-class has been loaded. <setup> (listing A.12 line 9-17): This section is also optional. It can be used
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 107 to define a method that will be invoked after the constructor has been called. It supports the definition of the method-name and further parameters whose syntax is equal to other parameter sections (see section A.3.7 on page 111). 1 <component> 2 <c l a s s >o r g . s e r v i c e s . S e r v i c e P r o v i d e r I m p l</ c l a s s > 3 <c o n s t r u c t o r > 4 <parameter> 5 <type>S t r i n g </type> 6 <v a l u e >MainProvider</v a l u e > 7 </parameter> 8 </ c o n s t r u c t o r > 9 <s e t up > 10 <method> 11 <name>s e t u p</method> 12 <parameter> 13 <type>S t r i n g </type> 14 <v a l u e >MainProvider</v a l u e > 15 </parameter> 16 </method> 17 </se t up > 18 </component> Listing A.12: Description of a component So far all files had been discussed that define the main configuration, the testcases and the involved components. The following sections describe how these components can be put together. A.3.5 The Connection Files These connections can be grouped. These groups of connections can be accessed via a unique id. In this example the assignment is done in the connection.xml see Listing A.13 on page 108. <path> (listing A.13 line 2): Defines the relative path where the connection files can be found. <connection> (listing A.13 line 3-6): These sections assign the XML con- nection file (listing A.13 line 5) to a unique id (listing A.13 line 4). The connection files are a little bit more complex. Defining these files may imply a big effort, depending on how many components are involved.
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 108 1 <c o n n e c t i o n l i s t > 2 <path>c o n n e c t i o n s</path> 3 <c o n n e c t i o n > 4 <id >b a s i c </id > 5 < f i l e >d i r e c t . xml</ f i l e > 6 </c o n n e c t i o n > 7 <c o n n e c t i o n > 8 <id >u s i n g −c o n t r a c t s </id > 9 < f i l e >u s i n g c o n t r a c t s . xml</ f i l e > 10 </c o n n e c t i o n > 11 </ c o n n e c t i o n l i s t > Listing A.13: The connections config file 1 <c o n n e c t i o n s > 2 <c o n n e c t i o n > 3 <id >p r o v i d e r −echo</id > 4 <c l i e n t > 5 <type>component</type> 6 <id >p r o v i d e r</id > 7 <hook> 8 <method>r e g i s t e r S e r v i c e </method> 9 <r e t u r n t y p e >v o i d</r e t u r n t y p e > 10 <p a r a m e t e r s > 11 <parameter> 12 <name>s e r v i c e </name> 13 <type>o r g . s e r v i c e s . S e r v i c e </type> 14 </parameter> 15 </p a r a m e t e r s > 16 </hook> 17 </ c l i e n t > 18 <s u p p l i e r > 19 <type>component</type> 20 <id >echo</id > 21 </ s u p p l i e r > 22 </c o n n e c t i o n > 23 <c o n n e c t i o n > 24 <id >p r o v i d e r −c o n c a t</id > 25 <c l i e n t > 26 <type>component</type> 27 <id >p r o v i d e r</id > 28 <hook> 29 <method>r e g i s t e r S e r v i c e </method> 30 <r e t u r n t y p e >v o i d</r e t u r n t y p e > 31 <p a r a m e t e r s > 32 <parameter> 33 <name>s e r v i c e </name> 34 <type>o r g . s e r v i c e s . S e r v i c e </type> 35 </parameter> 36 </p a r a m e t e r s > 37 </hook> 38 </ c l i e n t > 39 <s u p p l i e r > 40 <type>component</type> 41 <id >c o n c a t</id > 42 </ s u p p l i e r > 43 </c o n n e c t i o n > 44 </ c o n n e c t i o n s > Listing A.14: Connections file with direct connections
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 109 Connection files consist of several connection-parts ( see listing A.14 line 2-22 and 23-43). Each connection must be described by a connection-part which consists of an id, an involved client and a supplier. 1 ... 2 <c o n n e c t i o n > 3 <id >echo−c o n t r a c t echo−s e r v i c e </id > 4 <c l i e n t > 5 <type>c o n t r a c t</type> 6 <id >echo−c o n t r a c t</id > 7 <hook> 8 <method>r e g i s t e r S e r v i c e </method> 9 <r e t u r n t y p e >v o i d</r e t u r n t y p e > 10 <p a r a m e t e r s > 11 <parameter> 12 <name>s e r v i c e </name> 13 <type>o r g . s e r v i c e s . S e r v i c e </type> 14 </parameter> 15 </p a r a m e t e r s > 16 </hook> 17 </ c l i e n t > 18 <s u p p l i e r > 19 <type>component</type> 20 <id >echo</id > 21 </ s u p p l i e r > 22 </c o n n e c t i o n > 23 <c o n n e c t i o n > 24 <id >p r o v i d e r −echo−c o n t r a c t</id > 25 <c l i e n t > 26 <type>component</type> 27 <id >p r o v i d e r</id > 28 <hook> 29 <method>r e g i s t e r S e r v i c e </method> 30 <r e t u r n t y p e >v o i d</r e t u r n t y p e > 31 <p a r a m e t e r s > 32 <parameter> 33 <name>s e r v i c e </name> 34 <type>o r g . s e r v i c e s . S e r v i c e </type> 35 </parameter> 36 </p a r a m e t e r s > 37 </hook> 38 </ c l i e n t > 39 <s u p p l i e r > 40 <type>c o n t r a c t</type> 41 <id >echo−c o n t r a c t</id > 42 </ s u p p l i e r > 43 </c o n n e c t i o n > 44 ... Listing A.15: A connection that uses a Contract <id> (listing A.14 line 3): This defines the id that is used to access the fol- lowing connection definition. <client> (listing A.14 line 4-17): This section describes the client. It consist of several tags that are needed to completely define the client of a connection including its type, the Hook-Up method and the used parameters.
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 110 • <type> (listing A.14 line 5): Specifies if the client is a component or a Contract. This is necessary because Contracts and components have different meaning and behave differently during the test. and there are therefore different factories and managers to monitor their behavior and they must be notified about the connected objects. • <id> (listing A.14 line 6): Specifies the id of the component or Con- tract that is used as client. The id must be assigned either in the com- ponents description or in the Contract description. • <hook>(listing A.14 line 7-16): This section describes the method (Hook-Up) which is used to connect the client and the supplier. – <method> (listing A.14 line 8): The name of the method – <returntype> (listing A.14 line 9): : The expected return type. – <parameters> (listing A.14 line 10-14): : The parameter type of the Hook-Up-method. <supplier> (listing A.14 line 18-21) • <type> (listing A.14 line 19): Defines if the supplier is a component or a Contract. • <id> (listing A.14 line 20): The id of the instance that is used as supplier. Listing A.14 shows connections only between components. Connections between Contracts and components can be formulated in a similar way. Listing A.15 shows a connection between a Contract and components. This differs only in the definition of the <type> in the <client> and <supplier>. (Compare lines 5,19 and 26,40 in listing A.14 with the equal lines in listing A.15). Following the changes of the object type also the ids are adjusted to make sure that the right objects are connected. This flexible configuration allows different useful combinations of object connec- tions. If necessary connections of objects via several monitoring contracts can be implemented.
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 111 A.3.6 The Contract Files Before using Contracts in CrashIt , these have to be described in proper files: In one file, that maps an unique id to a contract(see listing A.17), and in another file, that describes the contract (see listing A.16). 1 <c o n t r a c t > 2 <c l a s s >o r g . s e r v i c e s . S e r v i c e C o n t r a c t</ c l a s s > 3 <c l i e n t t y p e s > 4 <t y p e name=”Hook”>o r g . s e r v i c e s . S e r v i c e P r o v i d e r </type> 5 </ c l i e n t t y p e s > 6 <s u p p l i e r t y p e s > 7 <t y p e name=”HookUp”>o r g . s e r v i c e s . S e r v i c e </type> 8 </ s u p p l i e r t y p e s > 9 </ c o n t r a c t > Listing A.16: Definition of the service-Contract Multiple instances of the same contract can be defined by assigning another id to this contracts, as it is done in Listing A.17. 1 <c o n t r a c t s > 2 <c o n t r a c t > 3 <id >echo−c o n t r a c t</id > 4 < f i l e >s e r v i c e −c o n t r a c t . xml</ f i l e > 5 </ c o n t r a c t > 6 <c o n t r a c t > 7 <id >c o n c a t −c o n t r a c t</id > 8 < f i l e >s e r v i c e −c o n t r a c t . xml</ f i l e > 9 </ c o n t r a c t > 10 </ c o n t r a c t s > Listing A.17: Definition of the used Contracts A.3.7 Creating Objects - the Parameter-Converter Several sections need the definition of objects by a given type and a certain value. As a consequence it was useful to define a general factory for those cases. This factory creates wrappers for initialized objects. These wrappers (converters) sup- port returning the object and its corresponding class. For some developers this may sound useless as the class of an instance can always be returned by invoking the .getClass() method, but in some cases - especially when simple types are used - this general factory is necessary. Another reason for implementing a general factory
  • APPENDIX A. THE USAGE OF CRASHIT - A SIMPLE EXAMPLE 112 was that only one factory should be responsible for initializing objects and conver- ters, because this allows to use only one configuration that stores for all supported object-types. Which converters are supported by the framework can be defined in a simple property file. 1 i n t=o r g . d i n o p o l i s . c r a s h i t . u t i l . c o n v e r t e r s . I n t C o n v e r t e r 2 S t r i n g=o r g . d i n o p o l i s . c r a s h i t . u t i l . c o n v e r t e r s . S t r i n g C o n v e r t e r 3 O b j e c t=o r g . d i n o p o l i s . c r a s h i t . u t i l . c o n v e r t e r s . O b j e c t C o n v e r t e r 4 Component=o r g . d i n o p o l i s . c r a s h i t . u t i l . c o n v e r t e r s . ComponentConverter 5 Boolean=o r g . d i n o p o l i s . c r a s h i t . u t i l . c o n v e r t e r s . B o o l e a n C o n v e r t e r Listing A.18: Converter property file The keys used in this property file can be used in the different configuration files to define the needed type (See line 10-15 Listing A.11 on page 104). A.3.8 Flowcontrol The section A.3.2 on page 100 describes the main usage of the Flowcontrol-mecha- nism. The syntax of the used tags may be a bit unusual, but it allows an effective parsing and setting up of the Flowcontrol sequences. It has not been designed to be written by hand, but it has been designed to facilitate the implementation of a graphical tool which allows to define and write these configuration-files. This tool will be a part of the next version of CrashIt . The files in this example, which uses Flowcontrol define only simple sequences that include one <if>...<then>...<else> construct. In CrashIt through Flowcontrol it is able to evaluate also <while> statements, nested statements and nested ex- pressions. Even <or> and <and> tags can be used to express a condition Designing such complex Flowcontrol expressions may be usefully in some cases, but generally, simple expressions should be preferred as they are more comprehensible.
  • Bibliography [Ale01] Andrei Alexandrescu. Modern C++ Design, Generic Programming and Design Patterns Applied. The C++ In-Depth Series. Addison Wesley, 2001. ISBN: 0 201 70431 5. [AM01] Crupi John Alur, Deepak and Dan Malks. J2EE Patterns. SUN Mi- crosystems Press, 2001. [Apa03] Ant development manual, 2003. http://ant.apache.org/manual/ develop.html. [ASU99] Alfred V. Aho, Ravi Sethi, and Jeffrey D. Ullmann. Compilerbau Teil 1. Oldenbourg, 1999. ISBN: 3 486 25294 1. [BBM+ 78] Barry W. Boehm, J.R. Brown, G. McLeod, Myron Lipow, and M. Merrit. Characteristics of Software Quality. TRW Series of Software Technology. North-Holland Publishing Co., July 1978. [Bec00] Kent Beck. eXtreme Programming explained. The XP Series. Addison- Wesley, first edition, October 2000. [Coc02] Alistair Cockburn. Agile Software Development. Addison-Wessley, 2002. [GHRV95] Erich Gamma, Richard Helm, Johnson Raplh, and John Vlissides. De- sign Patterns, Elements of Reusable Object Oriented Software. Ob- ject Oriented Technology. Addison Wesley, Massachusetts, 1995. ISBN: 0 201 63361 2. I
  • BIBLIOGRAPHY II [Hoa72] C.A.R Hoare. The quality of software. Software, Practice and Experience, 2(2):103–105, 1972. [JC00] Ann Jeffries, ron Anderson and Hendricksin Chet. eXtreme Programming installed. The XP Series. Addison-Wesley, 2000. ISBN: 0 201 70842 6. [Jon96] Cliff B. Jones. Systematic Software Development Using VDM. Prentice Hall International, Hemel Hempstead(U.K.), 1996. [jUn03] jUnit.org. junit homepage. available online http://www.junit.org, 2003. [Kra01] Reto Kramer. icontract - the java design by contract tool. http:// www.reliable-systems.com/tools/iContract/iContract.htm, 2001. visited, 09.08.2002. [McC77] James McCall. Factors in software quality. Technical report, General Electric, 1977. [Mey97] Bertrand Meyer. Object-Oriented Software Construction. Prentice Hall, Upper Saddle River, New Jersey, second edition, 1997. [ORF03] ORF. Software ist h¨ufigste Fehlerquelle. available online http:// a futurezone.orf.at/futurezone.orf?read=detail&id=157548, 2003. [Pop94] Sir Karl. Popper. Alles Leben ist Probleml¨sen. Piper, 1994. ISBN: o 3 492 22300 1. [Rom02] Ed Roman. Mastering Enterprise JavaBeans. Wiley, 2002. ISBN: 0 471 41711 4. [Sch] Klaus Schmaranz. AK-Softwareentwicklung - Programming for Large Libraries. Lecture notes. [Sch01] Klaus Schmaranz. Software-Entwicklung in C. Springer, July 2001.
  • BIBLIOGRAPHY III [Sch02] Klaus Schmaranz. Dinopolis - A Massively Distributable Component- ware System. Habilitation, July 2002. [Sut00] Herb Sutter. Exceptional C++. The C++ In-Depth Series. Addison Wes- ley, 2000. ISBN: 3 8273 1711 8. [Val03] Egon Valentini. Development of a framework for contract-based test- ing of software-components. Master’s thesis, IICM, Graz University of Technology, November 2003. available online http://crashit.hti.at.
  • List of Acronyms FTP . . . . . . . . . . File Transfer Protocol see also the glossary entry for FTP IDE . . . . . . . . . . . Integrated Development Environment J2EE . . . . . . . . . Java 2 Enterprise Edition RMI . . . . . . . . . . Remote Method Invocation see also the glossary entry for RMI URD . . . . . . . . . . User Requirements Document see also the glossary entry for URD XML . . . . . . . . . . eXtensible Markup Language see also the glossary entry for XML XP . . . . . . . . . . . . eXtreme Programming see XP IV
  • Glossary C: A high-level programming language developed by Dennis Ritchie and Brian Kernighan at Bell Labs in the mid 1970s. Although originally designed as a systems programming language, C has proved to be a powerful and flexible language that can be used for a variety of applications, from business programs to engineering. C++: A high-level programming language developed by Bjarne Stroustrup at Bell Labs. C++ adds object-oriented features to its predecessor C. C++ is one of the most popular programming language for graphical applications, such as those that run in Windows, UNIX, and Macintosh environments. Client: The inter-software-communication may be split into a side, which asks for a service, and the other side, which responds. The client represents that module, which requests services from the Supplier. Contract: defines stipulations between two parties, which communicates among themselves. The stipulations enclose obligations and benefits for both parties, the client and supplier. In CrashIt contracts are in concrete objects, that can be inserted between client and supplier. Contracts monitors the communication and guarantee, that stipulations hold. Framework: is a set of components building together an application. FTP: File Transfer Protocol The protocol used on the Internet for sending files. V
  • BIBLIOGRAPHY VI GOF : Gang of four, is an abbreviation of the four authors Gamma, Helm, Johnson and Vlissides that wrote the well known book Design Patterns, elements of reusable object-orientated software Hook: The hook pattern consists of the parts, the hook itself an the hook-up Hook-Up. The definition of the hook consists of two parts: the abstract hook and the concrete hook The abstract hook defines an interface for a component that represents methods how this component can be used. A concrete hook implements this abstract hook interface. In the example the Service interface is a abstract hook-up and the Echo class is a concrete hook because of the execute method that is used to ask the service. Hook Hook-Up pattern: The Hook Hook-Up pattern (refer to [Sch02]) is split in two parts, the Hook and the HookUp. Refer to section 4.3.1 on page 40 for a detailed explanation. Hook-Up: The hook-up part of the hook pattern can be split up into two parts: the abstract hook-up and the concrete hook-up. An abstract hook-up defines a method that is used to connect a concrete hook with an other component. A concrete hook-up is a component that implements the abstract hook-up inter- face. In the example the ServiceProvider interface is a abstract hook-up because it contains the hook-up method register(Service). The ServiceProviderImpl class is a concrete hook-up. Java: A high-level programming language developed by Sun Microsystems. It is an object-oriented language similar to C++, but simplified to eliminate language features that cause common programming errors. Java source code files are compiled into a format called bytecode, which can then be executed by a Java interpreter. Compiled Java code can run on most computers because Java interpreters and runtime environments, known as Java Virtual Machines
  • BIBLIOGRAPHY VII (VMs), exist for most operating systems, including UNIX, the Macintosh OS, and Windows. Module: The properties of a module can be summarized as follow: • A module has a clear defined functionality. • A module has a clear defined interface to external world. This interface can be described by functions, not by global variables. The interface integrates also a description of conditions to work properly. • A module is completely self-contained. It means, that no relation to outstanding code-parts is needed to work. • Modules are organized hierarchically following their abstraction-level RMI: Remote Method Invocation A set of protocols being developed by Sun’s JavaSoft division that enables Java objects to communicate remotely with other Java objects. Supplier: The inter-software-communication may be split into the part, which asks for a service, and the other part, which responds. The supplier represents the module, which responds on requests of the Client. Test: Consists in the validation, whether a product corresponds to the user re- quirements (see URD). Therefore testcases must be developed. In CrashIt these testcases can be grouped in Testcase-sequence, these in a Test- configuration. Testcase: specifies the method-call which is applied to a component, and the extended results. Testcase-sequence: specifies a sequence of testcases (see Testcase) to apply on a component. Test-configuration: specifies a sequence of Testcase-sequence.
  • BIBLIOGRAPHY VIII Unit test: Unit testing involves testing software code at its smallest functional point, which is typically a single class. Each individual class should be tested in isolation before it is tested with other units or as part of a module or application. The objective of unit testing is to test not only the functionality of the code, but also to ensure that the code is correct and robust. UNIX: A popular multi-user, multitasking operating systems developed at Bell Labs in the early 1970s. URD: User Requirements Document is a summary of all that requirements, that the customer demands from a product. A URD has to be so exact, that it can be used like a checklist to validate the final product. XML: eXtensible Markup Language A specification developed by the W3C6 . XML is a pared-down version of SGML, designed especially for Web documents. It allows designers to create their own customized tags, enabling the definition, transmission, validation, and interpretation of data between applications and between organizations. XP: eXtreme Programming, is a development method that forces documentation and includes testing as an implicit development task 6 http://www.w3c.org
  • Index A Description . . . . . . . . . . . . . . . . . . . . . 97 Available example CrashIt . . . . . . . . . . . . . . . . . . . . . . . . . 86 Flowcontrol . . . . . . . . . . . . . . . . . 100 File . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 B ResultStore Behavior class . . . . . . . . . . . . . . . . . . . . . . . . 99 Exception . . . . . . . . . . . . . . . . . . . . . . 15 result . . . . . . . . . . . . . . . . . . . . . . . 99 Observer . . . . . . . . . . . . . . . . . . . . . . . 40 test configuration . . . . . . . . . . . . . . 100 Behaviors Connection Mediator . . . . . . . . . . . . . . . . . . . . . . . 40 client.id . . . . . . . . . . . . . . . . . . . . 110 C client.type . . . . . . . . . . . . . . . . . . 110 Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 client . . . . . . . . . . . . . . . . . . . 109, 110 Coding standards . . . . . . . . . . . . . . . . . . . 69 connection . . . . . . . . . . . . . . . . . . . 107 Compatibility . . . . . . . . . . . . . . . . . . . 34, 36 hook-method . . . . . . . . . . . . . . . . . . 110 Components . . . . . . . . . . . . . . . . . . . . . 42–44 hook-parameters . . . . . . . . . . . . . 110 class . . . . . . . . . . . . . . . . . . . . . . . . . 106 hook-returntype . . . . . . . . . . . . . 110 constructor . . . . . . . . . . . . . . . . . . 106 id . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 setup . . . . . . . . . . . . . . . . . . . . . . . . . 106 path . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Contract . . . . . . . . . . . . . . . . . . . . . . . 44 supplier.id . . . . . . . . . . . . . . . . . . 110 Definition . . . . . . . . . . . . . . . . . . 30, 106 supplier.type . . . . . . . . . . . . . . . . 110 Dynamically loadable . . . . . . . . . . . 43 supplier . . . . . . . . . . . . . . . . . . . . . . 110 Exceptions . . . . . . . . . . . . . . . . . . . . . 43 Contracts Hook Hook-Up . . . . . . . . . . . . . . . . . 43 linear contracts . . . . . . . . . . . . . . . . . 23 Side-effect-free . . . . . . . . . . . . . . . . . . 42 nested contracts . . . . . . . . . . . . . . . . 24 components pseudo-linear contracts . . . . . . . . . 24 states . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Correctness . . . . . . . . . . . . . . . . . . . . . 33, 35 Configuration Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 components . . . . . . . . . . . . . . . . . . . . 98 CrashIt . . . . . . . . . . . . . . . . . . . . . . . . . . 2, 56 config . . . . . . . . . . . . . . . . . . . . . . . . . 98 Available . . . . . . . . . . . . . . . . . . . . . . . 86 connections . . . . . . . . . . . . . . . . . . . 98 Configuration . . . . . . . . . . . . . . 96–112 contracts . . . . . . . . . . . . . . . . . . . . . 99 Flowcontrol . . . . . . . . . . . . . . . . . . . 112 dir . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 Parameter Converter . . . . . . . . . . 111 resultchecker . . . . . . . . . . . . . . . . . 99 ResultStore . . . . . . . . . . . . . . . . . . . . . 99 testclasses . . . . . . . . . . . . . . . . . . . 99 Testcase . . . . . . . . . . . . . . . . . . . . . . . 104 User groups . . . . . . . . . . . . . . . . . . . . 57 IX
  • INDEX X D F Definition File Components . . . . . . . . . . . . . . . 30, 106 Configuration . . . . . . . . . . . . . . . . . . . 97 deterministic call . . . . . . . . . . . . . . . 15 Test Enhanced state-machine . . . . . . . . 22 configuration . . . . . . . . . . . . . . . . 100 nondeterministic call . . . . . . . . . . . 15 Flowcontrol . . . . . . . . . . . . . . . . . . . . . . . . 112 one way limitation . . . . . . . . . . . . . . 20 Configuration Result checker . . . . . . . . . . . . . . . . . 105 example . . . . . . . . . . . . . . . . . . . . . 100 Silent Catch . . . . . . . . . . . . . . . . . . . . 54 Sequence Test-Case . . . . . . . . . . . . . . . . . . . . . . . 58 example . . . . . . . . . . . . . . . . . . . . . 102 Test-class . . . . . . . . . . . . . . . . . . . . . . 104 short description . . . . . . . . . . . . . . 100 Description test configuration . . . . . . . . . . . . . . 100 Configuration . . . . . . . . . . . . . . . . . . . 97 test sequence . . . . . . . . . . . . . . . . . . 102 Test Framework . . . . . . . . . . . . . . . . . . . . . . . . . 30 configuration . . . . . . . . . . . . . . . . 100 Design by Contract . . . . . . . . . . . . . . . . . . 7 H Design Pattern . . . . . . . . . . . . . . . . . . 29, 40 Hook Hook-Up . . . . . . . . . . . . . . . . . . . . . 40 Hook . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 I Hook Hook-Up . . . . . . . . . . . . . . . . . 40 iContract . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Hook-Up . . . . . . . . . . . . . . . . . . . . . . . 41 Integration test . . . . . . . . . . . . . . . . . . . . . 51 Observer . . . . . . . . . . . . . . . . . . . . . . . 40 intermediate,working deterministic call states . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Definition . . . . . . . . . . . . . . . . . . . . . . 15 InternalFactors . . . . . . . . . . . . . . . . . . . . . 37 Development models Interoperability Extreme programming . . . . . . . . . . 68 Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Prototyping . . . . . . . . . . . . . . . . . . . . 68 Intuitivity . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Spiral model . . . . . . . . . . . . . . . . . . . . 66 Waterfall model . . . . . . . . . . . . . . . . 64 J Download Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 CrashIt . . . . . . . . . . . . . . . . . . . . . . . . . 86 jUnit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8–10 testcase . . . . . . . . . . . . . . . . . . . . . . . . . . 8 E testsuite . . . . . . . . . . . . . . . . . . . . . . . . . 8 Ease of use . . . . . . . . . . . . . . . . . . . . . . 34, 36 Eiffel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 M Enhanced state-machine Mediator Definition . . . . . . . . . . . . . . . . . . . . . . 22 Behaviors . . . . . . . . . . . . . . . . . . . . . . . 40 Exception . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Pattern . . . . . . . . . . . . . . . . . . . . . . . . . 40 Behavior . . . . . . . . . . . . . . . . . . . . . . . 15 Methodology . . . . . . . . . . . . . . . . . . . . . . . 63 Extendibility . . . . . . . . . . . . . . . . . . . . 33, 36 Modularity . . . . . . . . . . . . . . . . . . . . . . 34, 39 External factors . . . . . . . . . . . . . . . . . . . . 34 N Extreme Programming . . . . . . . . . . . . . . 68 Nested exceptions . . . . . . . . . . . . . . . . . . . 54 nondeterministic call
  • INDEX XI Definition . . . . . . . . . . . . . . . . . . . . . . 15 Correctness . . . . . . . . . . . . . . . . . 33, 35 Ease of use . . . . . . . . . . . . . . . . . 34, 36 O Extendibility . . . . . . . . . . . . . . . . 33, 36 Observer External factors . . . . . . . . . . . . . . . . 34 Behavior . . . . . . . . . . . . . . . . . . . . . . . 40 Internal Factors . . . . . . . . . . . . . . . . 37 Design Pattern . . . . . . . . . . . . . . . . . 40 Intuitivity . . . . . . . . . . . . . . . . . . . . . . 34 one way limitation Modularity . . . . . . . . . . . . . . . . . 34, 39 Definition . . . . . . . . . . . . . . . . . . . . . . 20 Performance . . . . . . . . . . . . . . . . 34, 37 OOP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Robustness . . . . . . . . . . . . . . . . . 33, 35 P Simplicity . . . . . . . . . . . . . . . . . . . 34, 38 Pair programming . . . . . . . . . . . . . . . . . . 69 states Pattern components . . . . . . . . . . . . . . . . . . . . 15 Mediator . . . . . . . . . . . . . . . . . . . . . . . 40 intermediate,working . . . . . . . . . . . 16 Performance . . . . . . . . . . . . . . . . . . . . 34, 37 Supplier . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 T Planning Game . . . . . . . . . . . . . . . . . . . . . 68 Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Prototyoping . . . . . . . . . . . . . . . . . . . . . . . 68 case R class . . . . . . . . . . . . . . . . . . . . . . . 104 Refactoring . . . . . . . . . . . . . . . . . . . . . . . . . 69 function . . . . . . . . . . . . . . . . . . . 105 Response time . . . . . . . . . . . . . . . . . . . . . . 49 parameters . . . . . . . . . . . . . . . . . 105 Result checker resultchechecker . . . . . . . . . . 105 Definition . . . . . . . . . . . . . . . . . . . . . 105 result . . . . . . . . . . . . . . . . . . . . . . 105 Result store configuration result . . . . . . . . . . . . . . . . . . . . . . . . . 99 flow-control . . . . . . . . . . . . . . . 100 ResultChecker . . . . . . . . . . . . . . . . . . . . . . 60 path . . . . . . . . . . . . . . . . . . . . . . . . 100 ResultStore Description . . . . . . . . . . . . . . . . . . 100 class . . . . . . . . . . . . . . . . . . . . . . . . . . 99 File . . . . . . . . . . . . . . . . . . . . . . . . . 100 Robustness . . . . . . . . . . . . . . . . . . . . . . 33, 35 Correctness . . . . . . . . . . . . . . . . . . . . . 47 Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Design principles . . . . . . . . . . . . . . . 51 Interoperability . . . . . . . . . . . . . . . . . 49 S Performance . . . . . . . . . . . . . . . . . . . . 48 Sequence Robustness . . . . . . . . . . . . . . . . . . . . . 48 example sequence Flowcontrol . . . . . . . . . . . . . . . . . 102 flow-control . . . . . . . . . . . . . . . 102 Service . . . . . . . . . . . . . . . . . . . . . . . . . . 31, 89 path . . . . . . . . . . . . . . . . . . . . . . . . 102 ServiceProvider . . . . . . . . . . . . . . . . . 31, 89 start-node . . . . . . . . . . . . . . . . . 102 shut down . . . . . . . . . . . . . . . . . . . . . . . . . . 79 testcases . . . . . . . . . . . . . . . . . . 102 Silent Catch Test-Case Definition . . . . . . . . . . . . . . . . . . . . . . 54 Definition . . . . . . . . . . . . . . . . . . . . . . 58 Simplicity . . . . . . . . . . . . . . . . . . . . . . . 34, 38 Test-class Software quality factors . . . . . . . . . 33–39 Definition . . . . . . . . . . . . . . . . . . . . . 104 Compatibility . . . . . . . . . . . . . . . 34, 36 Test-configuration
  • INDEX XII config dir . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 file . . . . . . . . . . . . . . . . . . . . . . . . . 98 W Waterfall model . . . . . . . . . . . . . . . . . . . . 64 typical steps . . . . . . . . . . . . . . . . . . . . 64