Master’s Thesis in Telematics
              for the Award of the Academic Degree
                          Diplom Ingenieu...
Diplomarbeit aus Telematik
             zur Verleihung des akademischen Grades
                          Diplom Ingenieur
...
Acknowledgments

There are a lot of people that had a strong influence on what I am and who are
therefore more or less resp...
iv



I hereby certify that the work reported in this thesis is my own and that work
performed by others is appropriately ...
Abstract

The IT community has learned how to design and implement highly reusable classes
during the past ten years. This...
Kurzfassung

Die IT-Gemeinde hat in den letzten 10 Jahren gelernt wie man vielf¨ltig wiederver-
                          ...
Table of Contents


Acknowledgments                                                                         iii


Abstract...
TABLE OF CONTENTS                                                                  viii



        3.1.3   Method-call mod...
TABLE OF CONTENTS                                                                     ix



5 Testing                     ...
TABLE OF CONTENTS                                                                     x



        7.3.1   Which way is th...
TABLE OF CONTENTS                                                                       xi



       A.2.2 Interfaces . . ...
List of Figures


 3.1   Model of a deterministic call . . . . . . . . . . . . . . . . . . . . . . . 17

 3.2   Model of a...
LIST OF FIGURES                                                               xiv



  8.2   UML diagram of the environmen...
Listings

 2.1   jUnit vector test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   9

 2.2   Implementation ...
LISTINGS                                                                         xvi



  A.14 Connections file with direct...
Chapter 1

Introduction

                    What would life be if we had no courage to attempt anything?
                ...
CHAPTER 1. INTRODUCTION                                                             2



requirements and which is extensi...
CHAPTER 1. INTRODUCTION                                                               3



teamwork. Egon describes in his...
CHAPTER 2. MOTIVATION                                                                  4




Chapter 2

Motivation

      ...
CHAPTER 2. MOTIVATION                                                                  5



      is the fundamental mista...
CHAPTER 2. MOTIVATION                                                                 6



In CrashIt an automatic test do...
CHAPTER 2. MOTIVATION                                                                7



will create the connections befo...
CHAPTER 2. MOTIVATION                                                               8



2.3      Related Work

As the top...
CHAPTER 2. MOTIVATION                                                                                       9



 1   pack...
CHAPTER 2. MOTIVATION                                                               10



Benefits and Weaknesses

We claim...
CHAPTER 2. MOTIVATION                                                                 11



      1. support of design for...
CHAPTER 2. MOTIVATION                                                            12



18
19     /* *
20        @pre ! isF...
CHAPTER 2. MOTIVATION                                                             13




As mentioned in the past sections...
CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS                                             14




Chapter 3

Aspects of calls a...
CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS                                        15



chosen language. For example, there...
CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS                                          16



store state information. Even whe...
CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS                                            17



   • circle: Equals a state.

 ...
CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS                                         18



result can be calculated, the comp...
CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS                                           19



Documentations are human readabl...
CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS                                            20



In this model transitions corre...
CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS                                          21



Sometimes a state should be reach...
CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS                                             22



Components which can be modele...
CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS                                          23



additional algorithms but others ...
CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS                                    24



   • There are different ways of reachin...
CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS                                       25



3.4.4    States and pre-conditions

...
CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS                                      26




                   Figure 3.10: A ca...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                           27




Chapter 4

General software q...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                                   28



It is common sense tha...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                           29



perts resulting in faulty prod...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                                     30



As the development-e...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                          31



based on the framework. This fr...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                                    32



       • boolean isIn...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                      33



  5. the services can be used by ap...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                         34



  4. Compatibility is the ease o...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                        35



Correctness

Correctness is the m...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                         36



   • Clear and simple design: th...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                          37



and living environments of the ...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                           38



Simplicity

Simplicity could b...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                        39



   • ability to understand user r...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                          40



4.3      Design patterns

Desig...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                      41



There are some other patterns that ...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                      42



4.3.2    Components

The idea of co...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                       43



Composition by Hook and Hook-Up : ...
CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS                                          44



Definition of the usage : Compon...
CHAPTER 5. TESTING                                                                   45




Chapter 5

Testing

          ...
CHAPTER 5. TESTING                                                                  46



ideas of reusable and extendable...
CHAPTER 5. TESTING                                                                    47



this must be carried out in an...
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Component based testing during the software development cycle
Upcoming SlideShare
Loading in …5
×

Component based testing during the software development cycle

1,086 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,086
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Component based testing during the software development cycle

  1. 1. Master’s Thesis in Telematics for the Award of the Academic Degree Diplom Ingenieur at the Graz University of Technology Component based testing during the software development cycle submitted by Gerhard Fliess Institute for Information Processing and Computer Supported New Media (IICM) Graz University of Technology September 2003 Supervisor: Univ.-Doz., Dipl.-Ing., Dr.techn. Klaus Schmaranz
  2. 2. Diplomarbeit aus Telematik zur Verleihung des akademischen Grades Diplom Ingenieur an der Technischen Universit¨t Graz a Komponenten basiertes Testen im Software-Entwicklungszyklus vorgelegt von Gerhard Fliess Institut f¨r Informationsverarbeitung und Computergest¨tzte neue u u Medien (IICM) Technische Universit¨t Graz a September 2003 Begutachter: Univ.-Doz., Dipl.-Ing., Dr.techn. Klaus Schmaranz Betreuer: Univ.-Doz., Dipl.-Ing., Dr.techn. Klaus Schmaranz
  3. 3. Acknowledgments There are a lot of people that had a strong influence on what I am and who are therefore more or less responsible that I was able to finish my studies by writing this thesis. First of all I’d like to thank my parents for supporting my education and providing the freedom that is necessary to make my own decisions. My personal development over the last years was strongly influenced by the flat-sharing community I was involved in and the friends I was sailing with. Also I want to thank all friends I made music with and those who shared many hours with me while discovering Graz at night. Thanks for your time and for discussing interesting topics sometimes late at night. At the university I have to thank Klaus Schmaranz for supervising this thesis and for holding sustainable courses that had a strong influence on my view of software- development. Special thanks to Egon Valentini for joining the team that enabled CrashIt and this thesis. Additional thanks goes to Astrid for being so patient during the final editorial steps of this work. Gerhard Fliess Graz, October 3, 2003 iii
  4. 4. iv I hereby certify that the work reported in this thesis is my own and that work performed by others is appropriately cited. Signature of the author: Ich versichere hiermit wahrheitsgem¨ß, die Arbeit bis auf die dem Aufgabensteller a bereits bekannte Hilfe selbst¨ndig angefertigt, alle benutzten Hilfsmittel vollst¨ndig a a und genau angegeben und alles kenntlich gemacht zu haben, was aus Arbeiten an- derer unver¨ndert oder mit Ab¨nderungen entnommen wurde. a a
  5. 5. Abstract The IT community has learned how to design and implement highly reusable classes during the past ten years. This was forced by writing down the different design patterns and defining the component based approach. Now another aspect becomes more and more important: do these classes behave like it has been defined during the design-phase? Existing test-benches are based on the approach that test-classes must be imple- mented or additional test code in the implementation has to be written. This code will include the same amount of errors than the implementation code so another type of defining test cases can be usefull to avoid this errors. Defining the tests in a descriptive way e.g. by writing down an XML file has the advantage that developers can specify the test in a languageindependent format that simplifies compatibility tests. These files describe what should be done during the test without implementing the neseccary steps. The files can be created before all features of the component that should be tested have been implemented. This thesis covers some theoretical topics on testing and software design and intro- duces a framework that is designed to fulfill the requirements of a modern software test-bench. v
  6. 6. Kurzfassung Die IT-Gemeinde hat in den letzten 10 Jahren gelernt wie man vielf¨ltig wiederver- a wendbare Klassen entwirft und implementiert. Das wurde durch die Definition der Design Patterns und das Aufkommen des komponentenbasierten Ansatzes verst¨rkt. a Nun wird ein weiterer Aspekt immer wichtiger, n¨mlich, ob sich die so entwickelten a Klassen wirklich so verhalten, wie es das Design vorgibt. Bestehende Testumgebungen verfolgen den Ansatz, dass spezielle Testklassen be- nutzt werden oder zus¨tzlicher Programmcode f¨r die Tests implementiert wer- a u den muss. Diese zus¨tzlichen Programmzeilen haben dieselbe Fehlerrate, wie der a eigentliche Programmcode. Darum muss ein anderer, weniger fehleranf¨lliger Weg a gefunden werden, Tests zu definieren. Die Definition der Testf¨lle in einer beschreiben- a den Art, z.B. als XML-Datei, hat den Vorteil, dass die Testf¨lle unabh¨ngig von der a a Sprache beschrieben werden. Dieser Ansatz erleichtert Kompatibilit¨tstests. Die a Testbeschreibungen k¨nnen dar¨ber hinaus schon erstellt werden, wenn noch nicht o u alle zu testenden Klassen implementiert sind. Sie beschreiben, was w¨hrend des a Tests zu tun ist, ohne die einzelnen Vorg¨nge zu implementieren. a Diese Arbeit behandelt einige theoretische Aspekte der Softwareentwicklung und des Testens und stellt ein Framework vor, das erstellt wurde, um die oben genannten Anforderungen an ein modernes Software-Testwerkzeug zu erf¨llen. u vi
  7. 7. Table of Contents Acknowledgments iii Abstract v Kurzfassung vi 1 Introduction 1 2 Motivation 4 2.1 Starting Point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Approach to a Solution . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.3.1 jUnit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.3.2 iContract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3 Aspects of calls and contracts 14 3.1 Method-calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.1.1 Types of calls . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.1.2 Components and States . . . . . . . . . . . . . . . . . . . . . 15 vii
  8. 8. TABLE OF CONTENTS viii 3.1.3 Method-call model . . . . . . . . . . . . . . . . . . . . . . . . 16 3.1.4 Sequence of calls . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.2 Contracts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.3 State-machines as sequence-models . . . . . . . . . . . . . . . . . . . 20 3.3.1 Limitations of state-machines . . . . . . . . . . . . . . . . . . 20 3.3.2 Enhanced state-machines . . . . . . . . . . . . . . . . . . . . . 22 3.4 The categorization of contracts . . . . . . . . . . . . . . . . . . . . . 22 3.4.1 Linear Contracts . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.4.2 Pseudo Linear Contracts . . . . . . . . . . . . . . . . . . . . . 23 3.4.3 Looped Contracts . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.4.4 States and pre-conditions . . . . . . . . . . . . . . . . . . . . . 25 3.4.5 How to combine components . . . . . . . . . . . . . . . . . . . 25 4 General software quality factors 27 4.1 A small example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.1.1 Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.1.2 Implementations and usage of the framework . . . . . . . . . . 32 4.2 Important Quality Factors . . . . . . . . . . . . . . . . . . . . . . . . 33 4.2.1 External factors . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.2.2 Internal Factors . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.3 Design patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4.3.1 Hook and HookUp . . . . . . . . . . . . . . . . . . . . . . . . 40 4.3.2 Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 4.4 Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
  9. 9. TABLE OF CONTENTS ix 5 Testing 45 5.1 Goals for running tests . . . . . . . . . . . . . . . . . . . . . . . . . . 46 5.1.1 Targets of test runs . . . . . . . . . . . . . . . . . . . . . . . . 47 5.2 Test-design concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 5.2.1 Test principles . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 5.2.2 Descriptive or declarative configurations . . . . . . . . . . . . 53 5.3 Exception safety and robustness . . . . . . . . . . . . . . . . . . . . . 54 6 CrashIt - A short introduction 56 6.1 What is CrashIt ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 6.2 Main concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 6.3 Testing topics covered by CrashIt . . . . . . . . . . . . . . . . . . . . 59 6.3.1 Correctness . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 6.3.2 Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 6.3.3 Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 7 Using CrashIt in the software design cycle 62 7.1 Methodology concepts . . . . . . . . . . . . . . . . . . . . . . . . . . 63 7.2 Development-models . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 7.2.1 The waterfall model . . . . . . . . . . . . . . . . . . . . . . . 64 7.2.2 The spiral model . . . . . . . . . . . . . . . . . . . . . . . . . 66 7.2.3 Prototyping . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 7.2.4 Extreme programming . . . . . . . . . . . . . . . . . . . . . . 68 7.3 Using CrashIt in different models . . . . . . . . . . . . . . . . . . . . . 71
  10. 10. TABLE OF CONTENTS x 7.3.1 Which way is the best? . . . . . . . . . . . . . . . . . . . . . . 71 8 The Implementation of CrashIt 73 8.1 Overall architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 8.2 Interfaces and design-decisions . . . . . . . . . . . . . . . . . . . . . . 75 8.2.1 The CrashIt environment . . . . . . . . . . . . . . . . . . . . . 75 8.2.2 Accessing the configuration, the configuration-layer . . . . . . 77 8.2.3 Using the framework . . . . . . . . . . . . . . . . . . . . . . . 78 8.2.4 Interfaces for test-cases . . . . . . . . . . . . . . . . . . . . . . 80 8.2.5 The XML-subsystem . . . . . . . . . . . . . . . . . . . . . . . 81 8.3 Available applications . . . . . . . . . . . . . . . . . . . . . . . . . . 83 8.3.1 Stand-alone Application . . . . . . . . . . . . . . . . . . . . . 84 8.3.2 Ant-task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 9 Conclusion 85 9.1 Personal experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 9.2 Project related topics . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 9.3 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 Appendix 88 A The usage of CrashIt - a simple example 88 A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 A.2 A service example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 A.2.1 Parts of the framework . . . . . . . . . . . . . . . . . . . . . . 89
  11. 11. TABLE OF CONTENTS xi A.2.2 Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 A.2.3 Implementations . . . . . . . . . . . . . . . . . . . . . . . . . 94 A.3 The CrashIt Configuration . . . . . . . . . . . . . . . . . . . . . . . . 96 A.3.1 The Configuration file . . . . . . . . . . . . . . . . . . . . . . 97 A.3.2 The Test-configuration File . . . . . . . . . . . . . . . . . . . 100 A.3.3 The Testcase files . . . . . . . . . . . . . . . . . . . . . . . . . 104 A.3.4 The Component Files . . . . . . . . . . . . . . . . . . . . . . . 106 A.3.5 The Connection Files . . . . . . . . . . . . . . . . . . . . . . . 107 A.3.6 The Contract Files . . . . . . . . . . . . . . . . . . . . . . . . 111 A.3.7 Creating Objects - the Parameter-Converter . . . . . . . . . . 111 A.3.8 Flowcontrol . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . III Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VIII
  12. 12. List of Figures 3.1 Model of a deterministic call . . . . . . . . . . . . . . . . . . . . . . . 17 3.2 Model of a nondeterministic call . . . . . . . . . . . . . . . . . . . . . 18 3.3 A one-way state-machine . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.4 State-machine for permutations of calls . . . . . . . . . . . . . . . . . 21 3.5 State-machine similar to a one-way state-machine . . . . . . . . . . . 21 3.6 An enhanced state-machine . . . . . . . . . . . . . . . . . . . . . . . 22 3.7 Linear contract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.8 A pseudo linear sequence . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.9 A looped sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.10 A call of a deterministic method . . . . . . . . . . . . . . . . . . . . . 26 3.11 A call of a nondeterministic method . . . . . . . . . . . . . . . . . . . 26 7.1 Elements of a methodology . . . . . . . . . . . . . . . . . . . . . . . . 63 7.2 The waterfall model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 7.3 The spiral model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 7.4 Life-cycle in XP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 8.1 CrashIt modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 xiii
  13. 13. LIST OF FIGURES xiv 8.2 UML diagram of the environment classes . . . . . . . . . . . . . . . . 77 8.3 UML diagram of internal configuration-interfaces . . . . . . . . . . . 79 8.4 UML diagram of the configuration tags (incomplete) . . . . . . . . . 83
  14. 14. Listings 2.1 jUnit vector test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2 Implementation of a stack enhanced by iContract . . . . . . . . . . . 11 8.1 CrashIt tag-library example . . . . . . . . . . . . . . . . . . . . . . . . 82 8.2 CrashIt as ant-task . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 A.1 Service interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 A.2 ServiceProvider Interface . . . . . . . . . . . . . . . . . . . . . . . . . 92 A.3 Configurator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 A.4 A sample application without a Configurator . . . . . . . . . . . . . 95 A.5 A sample application using a Configurator . . . . . . . . . . . . . . . 96 A.6 The main configuration file . . . . . . . . . . . . . . . . . . . . . . . . 98 A.7 A simple configuration without Flowcontrol . . . . . . . . . . . . . . 101 A.8 A Flowcontrolled configuration . . . . . . . . . . . . . . . . . . . . . . 101 A.9 A simple Testcase-sequence without Flowcontrol . . . . . . . . . . . . 103 A.10 A Testcase-sequence using Flowcontrol . . . . . . . . . . . . . . . . . 103 A.11 Calling a method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 A.12 Description of a component . . . . . . . . . . . . . . . . . . . . . . . 107 A.13 The connections config file . . . . . . . . . . . . . . . . . . . . . . . . 108 xv
  15. 15. LISTINGS xvi A.14 Connections file with direct connections . . . . . . . . . . . . . . . . . 108 A.15 A connection that uses a Contract . . . . . . . . . . . . . . . . . . . . 109 A.16 Definition of the service-Contract . . . . . . . . . . . . . . . . . . . . 111 A.17 Definition of the used Contracts . . . . . . . . . . . . . . . . . . . . . 111 A.18 Converter property file . . . . . . . . . . . . . . . . . . . . . . . . . . 112
  16. 16. Chapter 1 Introduction What would life be if we had no courage to attempt anything? Vincent van Gogh If somone is about to solve a problem by writing an application, it is important to write down the problem and work out the main aspects. After the problem has been specified in that way, it is useful to verify wether someone else has solved it before. Sometimes good solutions are found which can be easily adapted to solve the problem but not always do the available solutions cover all important aspects. This is the time to start your own implementation. I stumbled over the problem of testing applications that are implemented by a component-based approach during a lecture held byby Klaus Schmaranz at the Graz University of Technology. During a seminar-project two other colleques and I tried to solve it, but we had to stop at the point of definig the main aspects of this problem. We accepted as a result that no solution covers all our requirements or is exentsible enough to adapt it in a way that it can fulfill our requirements. One exciting year later the first prototype of a new test-bench was released. Egon Valentini and I had implemented it as a core-framework which fulfills most of our 1
  17. 17. CHAPTER 1. INTRODUCTION 2 requirements and which is extensible enough to form the basis of more complex test-benches. It provides a more general approach of defining test-cases than other frameworks and includes interfaces for implementing custom configurations which can thus be integrated into larger software development applications. Many people associate “testing” with proving that an application runs correctly. In- deed, testing is a destructive process with the aim of findinig bugs and thus crashing an application in a secure testing environment. Using this definition as a starting point, the created test tool is supposed to motivate programmers to detect more bugs and finally to create better programs. According to these ideas it was named CrashIt . Integrating the test into the design-process is one key-factor for a succesful test- process and therefore essential for a reliable and stable product. CrashIt simplifies the integration of the test-design into early phases of the development-cycle. There several approaches for solving the requirements of such test applications. However, no test-application which could accompany engineers through the whole design and implementation cycle of a software project exists. On suggestion of Klaus Schmaranz, Egon Valentini and I hence created a tool, that helps software developers to verify their designs and to test their implementations through an ex- tensive and complete check. It includes both unit- and integration-tests in a single session. Furthermore, we realized that the idea behind this test tool reveals a po- tential to change the status of testing in the IT community. The design is based on the idea to enhance the status of testing from a neglected to a widely recognised one. CrashIt was designed and implemented in teamwork by Egon Valentini and me (Ger- hard Fliess) between May 2002 and June 2003. The main modules of the framework were developed together. This and Egon’s theses ([Val03]) are the results of our
  18. 18. CHAPTER 1. INTRODUCTION 3 teamwork. Egon describes in his thesis the design of the framework, the module interconnections (refer to [Val03, chapter 6, p.62]) and the implementation of the main modules and Flowcontrol (refer to [Val03, section 5.1.3, p.49]). Futhermore it contains a general introduction into CrashIt , a critical evaluation of the project and takes a look on the future of this application. In addition, he explains the main modules and some use-cases [Val03, chapter 6, p.62]). I was responsible for the runtimeenvironment (also called CrashItEnvironment (see 8.2.1), a method to optimally describe tests through XML-Files and I implemented the modules that are used by CrashIt to generate test reports, i.e. I wrote all parsers and document generation classes. In my thesis I will discuss the usage of CrashIt in different software-design models as well as the principles of testing and explain the runtime-environment of this test tool. As these two theses cover the same software project both cover some equal topics. We shared some topics and wrote some general sections only once. This applies to chapters 1,4,5 and 6. Chapter 1 was written to equal parts by Egon and me, chapter 4 was mainly written by Egon except for the sections on design patterns and exceptions. Chapter 5 was written by me and chapter 6 includes only few parts of the complete chapter which can be found in Egon’s thesis. Klaus Schmaranz supervised our theses and we agreed on this proceeding after releasing the first version of CrashIt .
  19. 19. CHAPTER 2. MOTIVATION 4 Chapter 2 Motivation People forget how fast you did a job- but they remember how well you did it. Howard W. Newton 2.1 Starting Point During the development of an application some phases are more interesting and challenging than others. Most developers preferred designing and implementing an application than writing the documentations or installation manuals. The most condemned task is testing the application. The starting point which is to invite to have a critical look at testing, is excellently described by Kent Beck: Testing Strategy: Oh yuck. Nobody wants to talk about testing. Testing is the ugly stepchild of software development. The problem is, everybody knows that testing is important. Everybody knows they don’t do enough testing. [...] Remember the principle “Work with human nature, not against it.” That
  20. 20. CHAPTER 2. MOTIVATION 5 is the fundamental mistake in the testing book I’ve read. They start with the premise that testing is at the center of development. You must do this test and that test and oh yes this other one, too. If we want programmers and customers to write tests, we had better make the process as painless as possible, realizing that the tests are there as instrumentation, and it is the behavior of the system being instrument that everyone cares about, not the tests themselves. [...] Tests are most valuable when the stress level rises, when people are working too much, when human judgment starts to fail. So the tests must be automatic - returning an unqualified thumbs up/thumbs down indication of whether the system is behaving. [Bec00, Chap 18] In other words, Kent Beck explains that software engineering needs tools • to test parts of an application and whole applications automatically • resulting in the generation of a test report. Moreover, there is also a need for • a tool which developers may use during the whole life-cycle of a software-pro- ject, • computer-aided generation of test-cases from a specification and [Bec00] • testing using contracts [Mey97, Chap. 12-14]. 2.2 Approach to a Solution In order to provide a solution to this situation CrashIt was created. It consists of a Java-framework, which can automatically test Java-components on the basis of test-cases. After all test had be run it generates a test-report.
  21. 21. CHAPTER 2. MOTIVATION 6 In CrashIt an automatic test doesn’t cover the generation of test-cases. Automatic test means in this case that it runs the a test procedure composed of: 1. testing of single components 2. loading, connecting according components via Contracts and testing the root- component. (Contracts are used to check, whether conditions between two components hold during communication) 3. loading, combining components to a complete application and trying to find bugs, too. The user has to specify the test-cases before running a test. The goal of writing test- cases manually is that the test-engineer is able to get a feeling which tests are useful and which test are futile ones. For example, if a method computes the addition of two values, it would be useless to test this method for each permutation of these two values. Whereas appropriate tests consist in inspecting the method and trying to find those permutations where the addition fails, e.g. value1 = maxInteger and value2 = maxInteger, so if the method-result returned by an integer value, the addition of value1 and value2 produces an overflow. The goal of designing appropriate test-cases is understanding and scrutinizing a sys- tem in terms of correctness, robustness, extendibility, compatibility and performance (see also section 4.2 on page 33). CrashIt meets those premises by an extended pos- sibility how test-cases can be to formulated. It also provides a mechanism to group test-cases into simple or complex sequences, so called Testcase-sequences. Not only test-cases but also sequences of test-cases can crash a system. Moreover, CrashIt has the capability to organize unit- and integration-test in one test. Well designed systems can be split up into several parts, socalled modules that possibly consist of one ore more components. To perform a detailed test, each com- ponent and all possible combinations of them have to be checked. CrashIt can test all components at once and further run tests in connected components. The con- nections between the components must be specified in the configuration and CrashIt
  22. 22. CHAPTER 2. MOTIVATION 7 will create the connections before the test is started. A new approach to create modular software which is moreover correct and robust, consists in defining contracts between according components. This approach is also known under the synonym Design by Contract. The advantage of Design by Con- tract is that most components shrink, because through predefined conditions it can be taken for granted that some cases do not appear when the component will be processed. Supplementary checks can consequently be dropped. As a consequence, when the program decreases, it is easier keep an overview of its components and to maintain the code. (see also [Val03, chapter 4, p.34] and [Mey97]). During a test CrashIt supports Design by Contract through the possibility to insert Contracts between components. These Contracts are simple Java classes which had to fulfill some simple precepts that are caused by the fact, that Java do not support multiple inheritance from classes. Contracts have to: • implement the Contract interface • and to implement either the interfaces of both components • or they have to be derived from one component and implement the interface of the second one CrashIt is able to load contracts dynamically. It is possible to deliver them for addi- tional tests but in general they are not part of a packaged application. Finally, a very important requirement is the ability to compare test results with expected results. This is an easy task for simple result-types like integer. Whereas it fast raises to a complex job, if user-types have to be validated. The approach implemented in CrashIt consists of special Result-checker-classes. They are Java classes, that can be extended for own purposes. Result-checker-classes for simple types are already implemented.
  23. 23. CHAPTER 2. MOTIVATION 8 2.3 Related Work As the topic of writing stable and reliable software is getting more and more impor- tant several approaches for testing and designing such applications are available. In some software-development models the importance of testing has been increased as it is an important part of the process, e.g. eXtreme Programming forces developers to implement tests as the first step in a development-cycle. CrashIt is therefore not the first approach to support developers in writing stable and reliable applications. jUnit and iContract are two interesting projects that deal with these topics in different ways. 2.3.1 jUnit jUnit is a small but powerful framework for writing repeatable tests that has been designed by Erich Gamma and Kent Beck [jUn03]. It consists of two main classes: the TestCase and the TestSuite. Both implement the same interface Test that allows to start a test by invoking the method run. A TestCase performs one or more tests by invoking method calls and comparing the results of these calls with expected values. Concrete TestCases can be implemented by extending the class TestCase and by implementing methods whose names start with test. If a concrete TestCase class is applied to the framework, jUnit searches for those special named methods and invokes them. TestCases can be successful, cause an error or cause a failure in which case the test cannot be started. The result of a test run can be formatted by a TestFormatter and used as a report of the applied tests. A TestSuite can combine and start several TestCases. The method that is used to add a TestCase requires an instance of type Test and therefore a TestSuite can also combine several TestSuites. This simple design allows a flexible structure of test-classes. There are also methods for setting up and shutting down tests, so that it is possible to initialize needed environments for them.
  24. 24. CHAPTER 2. MOTIVATION 9 1 package j u n i t . s a m p l e s ; 2 3 import j u n i t . framework . ∗ ; 4 import j a v a . u t i l . V e c t o r ; 5 6 public c l a s s V e c t o r T e s t e x t e n d s TestCase { 7 protected V e c t o r fEmpty ; 8 protected V e c t o r f F u l l ; 9 10 public s t a t i c void main ( S t r i n g [ ] a r g s ) { 11 j u n i t . t e x t u i . TestRunner . run ( s u i t e ( ) ) ; 12 } 13 protected void setUp ( ) { 14 fEmpty= new V e c t o r ( ) ; 15 f F u l l = new V e c t o r ( ) ; 16 f F u l l . addElement (new I n t e g e r ( 1 ) ) ; 17 f F u l l . addElement (new I n t e g e r ( 2 ) ) ; 18 f F u l l . addElement (new I n t e g e r ( 3 ) ) ; 19 } 20 public s t a t i c Test s u i t e ( ) { 21 return new T e s t S u i t e ( V e c t o r T e s t . c l a s s ) ; 22 } 23 public void t e s t C a p a c i t y ( ) { 24 int s i z e = f F u l l . s i z e ( ) ; 25 f o r ( i n t i = 0 ; i < 1 0 0 ; i ++) 26 f F u l l . addElement (new I n t e g e r ( i ) ) ; 27 a s s e r t T r u e ( f F u l l . s i z e () == 100+ s i z e ) ; 28 } 29 30 public void t e s t C o n t a i n s ( ) { 31 a s s e r t T r u e ( f F u l l . c o n t a i n s (new I n t e g e r ( 1 ) ) ) ; 32 a s s e r t T r u e ( ! fEmpty . c o n t a i n s (new I n t e g e r ( 1 ) ) ) ; 33 } 34 public void t e s t E l e m e n t A t ( ) { 35 I n t e g e r i = ( I n t e g e r ) f F u l l . elementAt ( 0 ) ; 36 assertTrue ( i . intValue () == 1); 37 38 try { 39 f F u l l . elementAt ( f F u l l . s i z e ( ) ) ; 40 } catch ( ArrayIndexOutOfBoundsException e ) { 41 return ; 42 } 43 f a i l ( ” Should r a i s e an ArrayIndexOutOfBoundsException ” ) ; 44 } 45 public void t e s t R e m o v e A l l ( ) { 46 f F u l l . removeAllElements ( ) ; 47 fEmpty . r e m o v e A l l E l e m e n t s ( ) ; 48 a s s e r t T r u e ( f F u l l . isEmpty ( ) ) ; 49 a s s e r t T r u e ( fEmpty . isEmpty ( ) ) ; 50 } 51 52 } Listing 2.1: jUnit vector test This approach is simple to use and can be learned easily. Some development tools integrate wizards for defining and applying jUnit tests on applications. jUnit is widely used and there are different extensions for testing web- or J2EE (Java 2 Enterprise Edition) - applications. A small example which is part of the sample package of jUnit is shown in Listing 2.1 on page 9.
  25. 25. CHAPTER 2. MOTIVATION 10 Benefits and Weaknesses We claim that the sourcecode of test-classes contains a nearly equal amount of fail- ures, test-classes written in this way are as buggy as the original code. In a typical project at least as much test code as program source code will be written and there- fore this approach might be dangerous. It is true that the test-classes are often used, but how often will they be reviewed? A solution for this problem can be to define the tests in another way than program- ming test classes, e.g. in XML files. A further aspect is that it is more secure to specify what should be done in a test than to implement it. This approach may cause some limitations for the test design, but if the configuration possibilities are flexible enough, this should not be a real restriction. The definition of a test in a separated, non-Java source file also has the advantage that it can be created automatically or it can be the source for an automatically generated test report. jUnit is a well established framework with a huge amount of related projects. An- other advantage of jUnit is that it can be integrated easily into an IDE (Integrated Development Environment) as it has already been done for Eclipse1 . 2.3.2 iContract iContract is a Java tool that provides developers with support for Design by Con- tract(DbC). DbC refers to the concept of considering the interfaces between system components as contracts that are specified as integral parts of the sourcecode. Until today, the explicit specification of contracts by means of class invariants and mes- sage pre- and postconditions was available for Eiffel and some formal specification languages like VDM2 . iContract is a prototype tool that provides similar support for Java. It enables developers to take advantage of the following benefits: 1 Eclipse, see online http://www.eclipse.org 2 see online http://www.ifad.dk/Products/vdmtools.htm
  26. 26. CHAPTER 2. MOTIVATION 11 1. support of design for testability by enhancing the system’s observability (fail- ure occurs close to fault), 2. uniform implementation of invariant-, pre- and postcondition checks among members of your team, 3. documentation and code are always in sync and 4. semantic level specification of what requirements/benefits a class/message of- fers. iContract is a freely available sourcecode preprocessor, which instruments the code with checks for class invariants, pre- and postconditions that may be associated with methods in classes and interfaces. Special comment tags (e.g. @pre, @post) are in- terpreted by iContract and converted into assertion check code that is inserted into the sourcecode. The semantic of iContract includes also quantifiers (forall, exists) to specify properties of enumerations, implications, old- and return-value references in postconditions, as well as the naming of exception classes to throw. iContract supports the propagation of invariants, pre- and postconditions via inheritance and multiple interface implementation, as well as multiple interface extension mecha- nisms. Due to the non-mandatory nature of the comment tags, source code that contains DbC annotations remains fully compatible with Java and can be processed with standard Java compilers enabling a risk-free adoption of the technique. [Kra01] A small example: 1 /* * 2 @inv ( top >= 0 && top < max ) 3 */ 4 c l a s s MyStack 5 { 6 private O b j e c t [ ] e l e m s ; 7 private i n t top , max ; 8 9 /* * 10 @pre ( sz > 0) 11 @post ( max == sz && elems != null ) 12 */ 13 public MyStack ( i n t s z ) 14 { 15 max = s z ; 16 e l e m s = new O b j e c t [ s z ] ; 17 }
  27. 27. CHAPTER 2. MOTIVATION 12 18 19 /* * 20 @pre ! isFull () 21 @post ( top == $prev ( top ) + 1) && elems [ top -1] == obj 22 */ 23 public void push ( O b j e c t o b j ) 24 { 25 e l e m s [ top ++] = o b j ; 26 } 27 28 /* * 29 @pre ! isEmpty () 30 @post ( top == $prev ( top ) - 1) && $ret == elems [ top ] 31 */ 32 public O b j e c t pop ( ) 33 { 34 return e l e m s [−−top ] ; 35 } 36 37 /* * 38 @post ( $ret == ( top == max )) 39 */ 40 public b o o l e a n i s F u l l ( ) 41 { 42 return top == max ; 43 } 44 45 /* * 46 @post ( $ret == ( top == 0)) 47 */ 48 public b o o l e a n isEmpty ( ) 49 { 50 return top == 0; 51 } 52 } // End MyStack Listing 2.2: Implementation of a stack enhanced by iContract Benefits and Weaknesses iContract compiles pre-, postconditions and invariants into assertion-checks, that can be added to the original program. This additional code causes an application to slow down a little bit so that time-critical sections may crash. It could be a complex task whether conditions or invariants have to be specified for user types or a check implicates complicate calculations. The syntax of iContract is not as powerful as Java or Eiffel to permit unlimited possibilities to express condi- tions. A bad workaround would consist of specifying Java methods that implement the check. Furthermore these methods would be used by some statements in iCon- tract to express conditions. A benefit of iContract is the small implementation effort of conditions, because they can be simply expressed in some comment tags.
  28. 28. CHAPTER 2. MOTIVATION 13 As mentioned in the past sections both projects have its benefits and weaknesses. Both are interesting projects that are useful in many cases. However, they are not able to address all requirements that lead to the implementation of CrashIt . CrashIt is designed to combine unit- and integration- tests with the concepts of con- tracts. This was done in a way that allows a smooth integration of this concepts into existing development processes. One topic of this thesis is how CrashIt can be used and how it fits into different develoment-models This will be explained in later chapters and an extensive example of its usage can be found in the appendix (see page 88). The next sections will address some theoretical concepts in order to change developers view on software- design.
  29. 29. CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 14 Chapter 3 Aspects of calls and contracts If you be in a hurry, take the longer way. Lao Tse Sometimes it makes sense to lean back and think of concepts that are used over and over again to get a new view on them. Even reinventing things often opens a new angle and helps to gain a deeper understanding of the related topics. This chapter tries to open a new view on method-calls and contracts to facilitate developers in making their design decisions. 3.1 Method-calls Calling a method of a component is a very basic process, but it is not as simple as it seems. Each method must be called by using the correct parameters and will often return a value or an object reference. There are a lot of possible reasons why a call may cause an error. Some of these reasons, like an incorrect signature, can be checked by the compiler. Other reasons cannot be checked by the compiler because it is not possible to specify them in the
  30. 30. CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 15 chosen language. For example, there can be dependencies in the used parameters as there are two integer parameters and the first parameter must be greater than the second one. Other reasons for an error during a call can be a busy device, an incorrect initialized object, a reference to a deleted object etc. The only thing a developer can do is specify the behavior of the object, when the call cannot be finished successfully. The behavior is defined by an exception, so that it is possible to react to this error. Some methods are very simple or do not use some any critical resources, so that it is not possible for an error to occur during a call. 3.1.1 Types of calls According to this view there are two types of method-calls. Calls that will deter- mine successfully in either case and method-calls that can be interrupted because something is going wrong. To make it easier to distinguish between these types, they will be named deterministic call and nondeterministic call . Developers often concentrate on the deterministic calls and forget to define the be- havior of the nondeterministic ones. Both method types are equally important for the design process and also for testing the software. It is important to accept an exception as a valid result of a call. The difference between an exception and a result of a successful call is that the execution of the program will follow a different way. An invalid call would be if an error is not reported by an exception. This often causes a crash of the whole application. 3.1.2 Components and States One basic assumption is that each component has internal states. Developers often do not recognize these states and, consequently, there is no representation of these states in the program. Each component has at least two states, justCreated and deleted. Depending on the functionality there may be only these two states or a lot of other states between them. Sometimes flags or status variables are used to
  31. 31. CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 16 store state information. Even when they are implemented, they cannot be used for testing, as there can be a mistake in the coide that isz used to set the state infromation and everything seems to be right although an error has occurred. That is why the only reliable way of testing is to monitor the behavior of the class. Before describing the behavior of a component by defining a sequence of calls we should take a closer look at method-calls and define a model for them. 3.1.3 Method-call model Calling a method changes the internal state of a component. This can result in a new state or the component can stay in the same one. During the call the compo- nent changes into an intermediate state in which the component calculates the result of the method call. Sometimes it is necessary to consider this state, but often the component stays in this state for a very short time and therefore it can be ignored. When a method-call is blocking this state must be considered. A further example for considering this intermediate state is, when the method is solving a concurrency problem. In this case the method has to be defined assynchronized, which means that only one call of this method can be done as long as the execution of the method has not been finished. Another important point is that during the execution of a method an error can oc- cur. This will probably cause the component to switch into a state that cannot be directly reached from the original state. This seems to be a needlessly complicated view of method calls, but this model allows a simpler test-framework and it is im- portant to understand this view of method-calls to work with the test-framework. The test-framework becomes simpler because this model defines deterministic state- machines for this nondeterministic problem of calling methods. Here are some rules for understanding the following graphics and the more enhanced ones in later chapters.
  32. 32. CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 17 • circle: Equals a state. • line with arrow: Equals a transition between two states. The arrow shows the direction and points at the result state • solid line: Equals a normal transition. This transition is the reason why the method was written. • dotted items: Denote internal intermediate states or transitions that are used by the model. Transitions caused by events like exceptions must be specified for error handling in order to make it possible to test the correct behavior. • rectangle: Equals an event, like a method-call or the occurrence of an excep- tion. In section 3.4.5 we will see that an event can have more than one effect, however, up to now the only effect is that the transition is executed. Model of deterministic calls In this example the component is in the state A before a method is called. During the call the component is in the intermediate state A*. After the result has been calculated there is only one possible state, in this example labeled as B. Figure 3.1: Model of a deterministic call Model of nondeterministic calls A nondeterministic call is quite equal to a deterministic call except for the fact that the intermediate state A* has two possible transitions into further states. If the
  33. 33. CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 18 result can be calculated, the component will switch into state B. If an error occurs, the component will switch into the state C. Figure 3.2: Model of a nondeterministic call 3.1.4 Sequence of calls It is very important to describe the usage and the behavior of components both in order to use and to test the component. Thinking of the description of a component and considering the testability during the design process makes it easier to design simpler and therefore better components. If it is easy to test a component, it is probably easy to use it. Otherwise components that do not facilitate there tests often include design problems. The behavior and the allowed usage can be described by a sequence of method-calls and their results. Some methods cannot be called, when a certain state has not been reached. These calls define a possible sequence of method-calls which is the only way the component can be used. Sequences of method calls are the first step to define contracts of components. 3.2 Contracts When a component is implemented, there is no direct way to specify its usage and behavior. The usage is written down in the documentation of a component.
  34. 34. CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 19 Documentations are human readable, but it is difficult to write a program that extracts test-cases for the component from the documentation. A good way of solving this problem is to define contracts between components. These contracts can be written in a machine readable form and therefore it s possible to transform them into human readable texts or diagrams. One part of a contract is the way of calling methods of a component. Depending on the language exceptions may be thrown when an error occurs. Contracts can not solve the problem that has caused the exception, but they can specify a way of dealing with it. A correct signature cannot prohibit an illegal call of a method. Methods often need some pre-conditions which must be fulfilled before a method can be used. After a successful call the result has to match a post-condition and the state of the component may have changed. Contracts are defined by signatures and a possible sequence of method- calls Describing the correct sequence can be difficult, as there may be many variants. A good design should prevent complex sequences and limit the amount of possible variants. One good approach to describe the correct sequences is to define the start- and the end-state for each method-call. In some cases it is useful to consider the intermediate states. The description can be defined as set of states combined with a set of methods and transitions between them. • S set of states • Ss ⊂ S set of start states • Se ⊂ S set of end states • M set of methods • Ss × M → S e
  35. 35. CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 20 In this model transitions correspond to method calls. There are various types of contracts which can be classified by taking a look at the time line of permissible method-calls. This will be modeled by using a state-machine. 3.3 State-machines as sequence-models State-machines are well-known constructs to control different processes. They are defined by states and transitions between these states. [ASU99, p. 137]1 A state- machine does not remember the earlier states, it only knows the actual state. This behavior is not enough for modeling the allowed sequences of method calls, and there are other limitations, too, so that it is necessary to enhance the functionality of state-machines. 3.3.1 Limitations of state-machines Before defining a more enhanced state-machine we should discuss their limitations. The one-way-limitation Figure 3.3: A one-way state-machine A state-machine allows only one way of applying the transitions. There is no way to do parallel transitions. It would be better to specify the correct sequence of calls, but sometimes it is not possible to specify only one allowed way. Figure 3.3 shows a sequence of transitions that result in state Y. Reaching state Y in this example is only possible going from state A to state B, etc. 1 The page number referes to the german version.
  36. 36. CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 21 Sometimes a state should be reached when several other states have been reached without defining an order. State-machines that allow such sequences will quickly get too complex to handle. Figure 3.4: State-machine for permutations of calls Defining groups of states One solution to this problem could be building groups of states, but state-machines do not support groups. There must be policies which transitions are valid within the group and which cannot be applied. These policies will get less complex if they can be defined for state-machines as in figure 3.5. Figure 3.5: State-machine similar to a one-way state-machine In this state-machine all transitions that result in a new state can be called in this state again.
  37. 37. CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 22 Components which can be modeled by such state-machines consist of methods that can be called at least twice without causing an error. This might not apply for all methods but a lot of methods show this behavior. For instance, during the setup of a component it might be necessary to store references of other components by calling a method. This method can be called several times without causing an error. An example for a method that does not have such a behavior could be the opening of a file containing the setup information. Calling this method twice may cause an exception when the file is still opened. A robust implementation is to avoid such an exception and to make it possible to call the method more than once. 3.3.2 Enhanced state-machines Enhanced state-machines allow to test contracts as defined on page 19 by using a set of states. It supports a group of states. All transitions that are allowed between these states can be applied. A method that causes a switch to a new state but can not be called again in this state, can also be called once in these state-machines. After a state-machine has been defined, this enhanced functionality can be applied by grouping states. Figure 3.6: An enhanced state-machine 3.4 The categorization of contracts There are various categories of contracts. For implementing systems that deal with contracts it is important to define categories for these types. Some types do not need
  38. 38. CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 23 additional algorithms but others need loop-detectors or other algorithms based on graph-theory. 3.4.1 Linear Contracts Linear contracts are the simplest type. Each state has only one transition that ends in another state. Figure 3.7: Linear contract It is very simple to handle such contracts, as there is a clear way of how a component that supports this type of contracts has to be used. 3.4.2 Pseudo Linear Contracts This type is slightly more complex than the linear type. One or more states have one or more transitions that end either in the same state or in a state that has not been visited. Figure 3.8: A pseudo linear sequence Building a system that is able to handle such contracts must solve two problems:
  39. 39. CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 24 • There are different ways of reaching a state. • How often is a transition called that ends in the same state. 3.4.3 Looped Contracts Developers should avoid writing contracts that have this behavior, but sometimes it is necessary for a component to have such a contract. Figure 3.9: A looped sequence A system for these contracts must solve several problems: • There are different ways of reaching a state. • How often is a transition called that ends in the same state. • How to detect loops. • Finding all ways to reach a state. • Finding the shortest way to reach a state.
  40. 40. CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 25 3.4.4 States and pre-conditions Most models for testing software components use pre-conditions to define what is to be done before a component can be used. Pre-conditions are often defined in a very formal way resulting in lots of work to define them. States are defined during the design of a component. Using this view on components may result in a simple design and simple designs may cause less errors. In some way there is a relation between states and preconditions: • Methods that need the same pre-conditions are in the same state. • The summary of all states that are needed to reach a state can be interpreted as the pre-conditions of this state. It it seems possible to derivate most pre-conditions by using states. However, this has not been proven and evaluating this assumption and developing an algorithm should be part of further research. 3.4.5 How to combine components Components will be used in combination with other components. All these com- ponents have contracts that must be fulfilled. The components are connected by applying method-calls on each other. The component that applies a method-call on another component is called caller. As described in section 3.1.3 there are different types of method calls, determin- istic calls and nondeterministic calls. Figure 3.10 shows the model of combining components by applying deterministic calls. This is quite simple because invoking a deterministic call will always result in an end-state. Image 3.10 shows the model of a deterministic call between two compo- nents. Both involved components change into their end-states.
  41. 41. CHAPTER 3. ASPECTS OF CALLS AND CONTRACTS 26 Figure 3.10: A call of a deterministic method Figure 3.11: A call of a nondeterministic method Calling a nondeterministic method is more complex because there is more than one possible end-state. Both components must handle all possible exceptions and switch into the corresponding state (cf. 3.11).
  42. 42. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 27 Chapter 4 General software quality factors Start by doing what is necessary,then do what is possible, and suddenly you are doing the impossible. St. Francis of Assisi Software-quality is a controversial topic as creating applications are getting more and more complex. Developers have to manage the trade-off between fast development cycles in a complex environment and customers that need stable, reliable products. The situation is getting worse when applications are used by people that do not have a technical background. Technicians often find solutions that are logical and stable when they use them, as they know the processes in the background, the relations between them and the resulting operation of a certain device. People who do not have this background information - because it is not their main topic and they only want to use a device or an application - often get problems with these products because their usage is not intuitive and resulting bugs do not allow an efficient usage of the product. The problem of a not intuitive user-interface or behavior of a product can only be solved by usability tests during the design. Decreasing the rate of errors can be done during implementation, but the basis must also be established during the design.
  43. 43. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 28 It is common sense that it is impossible to write huge applications without bugs, but this should still be the goal during the development process. Developers may not lean back without struggling for reaching this goal because everybody says that it is not possible to reach it. Economical conditions force companies to decrease the development time and its costs but this is short-sighted because bugs in deliv- ered applications cause much bigger costs than bugs that are detected during the development process. It doesn’t matter if dishwasher, car, washing machine or mobile phone - if a device doesn’t work, there often is a bug in the software. [...] Experts quite agree that software could be essentially more reliable if producers would force that. Further, they believe that a liability for bugs could speed this process up. Annually buggy software costs 60 bn. USD for the US-economy [...] There are no automatic test-tools1 : One half of their time developers devote to programming, the other one to searching bugs and patching these. In the test phase, a program reveals approximately 10 bugs per 1000 lines of code. The problem of developers consists in that there exists no robust tool to measure the reliability of their software-designs. [ORF03] Users have accepted that an application crashes once a day, not because they know about the complexity of the programm but for the reason almost every programm behaves in that way. As a consequence they have accepted this misbehavior as a bothering, unavoidable fact. As our life is getting more and more digitalized, more reliable software is needed. In the IT-world there is a lack of experts, so lots of projects are delegated to non-ex- 1 The term “automatic test-tools” does not mean that they can automatically generate test- cases, but they can automatically run test-cases. Automatically generated test-cases would never be as efficient as test-cases, manually created by intelligent inspection of source-code.
  44. 44. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 29 perts resulting in faulty products. By means of new and more powerful programming languages, tools for testing and formal specifications, the IT-community is trying to remedy this situation. This is not the final solution, but certainly a step into the right direction. One of the greatest “revolutions” in software design during the last few years has been both, the invention of the object-oriented approach and the introduction of the OOP quality factors, like reusability, modularity and extendibility (section 4.2 on page 33). The evolution of OOP temporarily ended in the definition of the Design Patterns which can be interpreted as the object-oriented counterparts of the established paradigms of structural languages. These patterns allow the implementation of components to fulfill the goals of the OOP approach. Christopher Alexander provides a good description of patterns: “Each pattern describes a problem which occurs over and over again in our environments, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice” [GHRV95, Chap. 1] This definition describes patterns in buildings and towns, but it can also be applied to OOP. Patterns define solutions for general problems using a range of known patterns in a very formal way. A very good book by Gamma, Helm, Johnson and Vlissides (GOF ) is Design Patterns, Elements of reusable object-orientated software. Section 1.1 [GHRV95, p.2] defines the term pattern by specifying four elements: the pattern name, the problem, a solution and its consequences. The book addresses 23 patterns that are grouped into 3 categories. All patterns are explained and further treated by discussing an example, so that it is easy to get an overview of all patterns. Knowing these patterns is not enough, they must be internalized to understand their deeper meanings and their potential. These 23 patterns are patterns for object-orientated development environments.
  45. 45. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 30 As the development-environment provides special possibilities, e.g. the J2EE2 - platform, existing patterns will be adapted3 or new ones will be invented4 (see [AM01]). In combination with the ideas of OOP a further definition is becoming popular, namely that of a component [Sch02] A component is represented by a class that concentrates on one topic and which is connected with other components via a Hook and a HookUp (see page 40 for details). Another aspect of modern languages is that they are very useful for writing correct and robust programs as they provide the chance to handle unexpected situations by defining a clear mechanism for notifying such situations. This can be done by defining exceptions which track faulty states of a program and guide the program to a defined end- or error-state. The aim of this chapter is to explain the terms which are needed in the following chapters including design patterns and software quality factors by means of a simple example5 . 4.1 A small example The example is implemented as a component-based framework6 allows to build string manipulation applications. The framework captures the main design and thus as- signs the responsibilities. It therefore dictates the architecture and allows the devel- oper to concentrate on the specific requirements of the concrete application which is 2 A good book on J2EE is Mastering Enterprise JavaBeans by Ed Roman [Rom02] 3 The command pattern has been adapted to J2EE see [AM01, chapter 7.2]. 4 The business delegate[AM01, chapter 8.1] was designed especially for J2EE 5 The entire example and the usage of CrashIt as its test bench can be found in the appendix. 6 A framework is a set of cooperating classes that are designed to support developers in creating applications for different domains [GHRV95, p. 26].
  46. 46. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 31 based on the framework. This framework is designed to build string manipulation applications. The framework defines Services that can be used by a ServiceProvider. A Service deals with strings on which it performs some computations by invoking the method String execute(String value). The framework consists of several classes, of which two are important for the further explanations: Service: it processes a string value and returns the modified value. A Service must be initialized, before using it. It is possible to check whether the service is initialized or not. A Service has a unique id, that serves ServiceProvider to address the this Service. ServiceProvider: it manages different Services. The ServiceProvider7 can ini- tialize all registered services. A service can be used by passing two parameters to the ServiceProvider, the id of the service and the value that should be processed. 4.1.1 Interfaces There are different interfaces that correspond to the parts of this framework. In this section the interfaces are shortly described a detailed documentation can be found in Appendix A.2.1 on page 89. The service interface (see listing A.1) consists of four methods: • String getServiceId(): returns the id of the concrete Service • String execute(String) throws ServiceException: performs a calcula- tion on a value • void init() throws ServiceException: initializes the service 7 The ServiceProvider can be interpreted as Mediator [GHRV95, Chap. 5].
  47. 47. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 32 • boolean isInitialized(): returns true if the service has been initialized The service-provider interface (see listing A.2) is defined by8 : • void registerService(Service) throws ServiceException: registers an uninitialized Service • void initServices() throws ServiceException: initializes all services • String ask(String id, String value) throws ServiceException: forces the service that corresponds to the id to process the assigned value 4.1.2 Implementations and usage of the framework In this example two simple services are implemented: Echo: returns the given value Concat: concatenates the given value with the previously stored values (The first time Concat is called, it returns the unchanged value.) 9 These two services can consequently be registered at the ServiceProvider and addressed by it. This can be done by implementing the following steps: 1. an instance of a ServiceProvider must be created 2. both services are loaded 3. an external object registers the services with the provider and 4. the provider initializes these registered services 8 A smarter implementation of this components would go beyond the scope of this example, but would be necessary for real world applications. 9 This implementation is done in the class org.service.ServiceProviderImpl that is part of the CrashIt example. The whole example is part of the CrashIt distribution
  48. 48. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 33 5. the services can be used by applying the corresponding key and a value An example of an application that uses the framework in the described way can be found in Listing A.4 on page 95. Services and ServiceProvider will be connected at runtime which allows a very flexible configuration and extension of an application that uses this framework. If this application must be modified, only its configuration has to be updated without recompiling the framework. For instance, the implemen- tation used in the appendix supports a Configurator class. It is responsible for setting up the ServiceProvider. Configurator parses a configuration file, where the used services are specified. In this version services can be added during the start up. A smarter implemen- tation would be able to add services during execution, but this example is designed to clear the view on the basic ideas and not to provide a perfect solution to the problem. Before starting to explain the remaining terms, the following sections deal with achievable software quality factors. 4.2 Important Quality Factors The premises, that must be fulfilled to produce high quality software which the economy demands, can be grouped into these external factors : 1. Correctness is the exact representation of a software product with respect to its specification. 2. Robustness is the ability of software to respond appropriately to abnormal or modified conditions. 3. Extendibility is the ease of extending software products to new changes of specification or other problem domains.
  49. 49. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 34 4. Compatibility is the ease of combining software elements with others. 5. Ease of use is the ease with which people of various backgrounds and qualifi- cations can learn to use software products and apply them to solve problems. 6. Performance is the capability to predict the time exposure of an applica- tion depending from the software architecture, amount of processed data and hardware and in following internal factors: 1. Simplicity is the ability to model real world problems in intuitive software structures. [Bec00, Chap 17] 2. Modularity is the ability to decompose an application in single modules, which communicates only over defined interface. [Sch, Chap 2.4] 3. Intuitivity is the capability to write the code so that other developers may read it like a book, this without comments in the code. [Sch01] External factors define characteristics of a program determined by users, whereas internal factors concern software engineers. They have access to the software design and its sourcecode. The following sections explains in detail these factors. Further external and internal factors are explained by [Mey97], [Ale01] and [McC77]. 4.2.1 External factors The journalist of the article [ORF03] tries to picture out, why many software pro- ducts are not reliable (neither correct, nor robust). As an outstanding user, he judges the actual situation in the software industry. This judgment from external users equals the definition of the external factors. In this sections the most important external factors with respect to CrashIt will be discussed.
  50. 50. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 35 Correctness Correctness is the main and prime quality of a software product; no other factor has such a weight like this. The best user interface counts nothing if the software is not correct. The correctness is theoretically derived from the exact implementation of the URD (User Requirements Document). But also the URD must underly correctness and completeness. For little systems an exact URD is possible, but for huge applications like an operating system, the numerous user requirements for the software product could easily conflict among themselves. To create reliable software, the correctness already starts when user requirements have to be declared and verified on the client’s desires. Correctness finishes with the maintenance of the product. A method which helps to employ correctness is the layered structure of software: The software can be split into more clearly defined and limited layers. Each layer relies on lower ones. The process to proof the correctness of a layered system is a incremental process. That means that the correctness of a layer depends on the correctness of lower layers. If, for example, a system makes use of libraries then firstly it must be ensured that these are correctly implemented. Relying on this as- sumption it is possible to test the correctness of the system. The basic layer should be selected evaluating user requirements, operating system, language and compiler, hardware, running environment and user groups. Design patterns may help to find the layers and interfaces between them. [Hoa72], [BBM+ 78] Robustness Correctness is the exact representation of the specification, whereas robustness also expresses how a system reacts in situations which are not include in the specification. The strength of robustness consists of the capability to notice failures, to guide the system into a secure state or to catch the error and start some recovery processes. To grant robustness, following points help:
  51. 51. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 36 • Clear and simple design: thereby it is easier e.g. to locate errors and to react accordingly • Formal specifications, see [Jon96] • Start testing already during specification phase • Not only valid tests should be formulated, but also invalid tests, which should verify the robustness Extendibility An essential principle for improving extendibility is modularity: The better a system can be split into autonomous parts, the higher the likelihood that a change in the system would only affect one or some modules, rather than the whole system (further details are explained in section Modularity on page 39). Compatibility A commonly known situation: A new operating system is installed and some old applications would not run anymore. This is a common issue of not considered compatibility. Compatibility is the factor which implies the most work: Depending on the size and differences of a user group, this factor demands to consider each different environment, where the user would use the new application. Also for this factor, layered program structure could help to move all user or system relevant varieties into one or less layers. Using this technique, only some less layers would be affected by compatibility. Of course, if the specification is not good enough, the migration to other platforms is quite impossible. Ease of use The factor ease of use should just be considered when user requirements are to be defined. The more software designers understand and study the desires, backgrounds
  52. 52. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 37 and living environments of the user group, the more user requirements can be ex- pressed through a simple and clear structure. Upon user requirements, the software design is developed. The simpler and clearer these requirements are, the easier the process of development becomes and the more uncomplicated and unproblematic the structure of software can be defined. Simplicity and clarity are premises for this factor. Performance This important factor would be ignored, if for example the calculation of tomorrow’s weather takes more than 24 hours. Mostly, the performance of a system correlates with the hardware capabilities, but also with the complexity of a program and how much data types will be abstracted (The abstraction of types means its generaliza- tion. This has the disadvantage, that special types must be derived from general ones. This process causes performance costs). The software community shows two typical attitudes: • Either a program is designed with respect to speed omitting intuitiveness and simple design (this program would be so specialized, that it is not open for reusability or extendibility) • Or it is designed with respect to generality: design a program as general as possible to be used for many different problem domains, without dealing with process time or other limited resources. 4.2.2 Internal Factors Internal factors describe qualities for software designers. These factors are in general the premise for external factors: correctness, robustness, extendibility, compatibility and ease of use based on simplicity and modularity.
  53. 53. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 38 Simplicity Simplicity could be one of the hardest things in peoples’ lives: It seems crazy, but sometimes it is easier to do something more complicated than to do things simple. This is particularly true, when you have been suc- cessful doing the complicated thing in the past. Learning to see the world in the simplest possible terms is a skill and a challenge. The challenge is that you may have to change your value system. [Bec00, Chap 24] Most people correlate quality work to simplicity. For example a car: it has few com- mand devices, but they are usefully arranged, so that driving is a simple task. The manufacturers of cars hide complex structured engines behind simple to use devices. A gimmick to achieve simplicity in software: first avoid complexness, secondly hide complex processes behind simple-to-use devices. Simplicity will not only influence the external factors, but also: • A simple design is easier to communicate with than a harder one. • Simplicity invokes user to adopt simple strategies to achieve simple and fur- thermore also correct, robust and extendible programs • Simple design gives quicker, more and exact feedback about the quality of the project. Principles to improve simplicity are: • develop software using a modular approach and Design Patterns (see [GHRV95]) • small initial investment in the design • incremental change: The strategy of simplicity in design will work by gradual change • be open for extensions and changes in the specification
  54. 54. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 39 • ability to understand user requirements, their background and living environ- ment. Only on this way, software developers are able to apply correctly user desired software solutions. A further key to support simplicity is uniformity in the programming style (e.g. using some style guides). Another key is of course intuitivity, so that a concept of a system is understandable and logic for colleagues in the developing team. Finally each developer should keep in mind, that a system has to be maintained. It is therefore useful to get the habit to write exact documentation, furthermore to think and develop in simple terms. Modularity A software construction method is modular, then, if it helps designers pro- duce software systems made of autonomous elements connected by a coherent, simple structure. [Mey97, Chap. 3] A modular design method could be classified using this five criterions: • Modular decomposability: a system can be split into less complex and rela- tively autonomous parts that communicate over a simple structure • Modular composability, is connected to reusability: it represents the capability to construct new systems using existing modules. Only the communication structure has to be designed. • Modular understandability: each module can be read and understood by itself, without knowing other or only few modules. • Modular Continuity: an extension in a module does not trigger changes in other modules. • Modular Protection: the consequences of an error caught or caused by a mo- dule are limited to that module or at worst propagate to few neighboring modules.
  55. 55. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 40 4.3 Design patterns Design pattern describes a problem, that often occurs in our environment, and gives a solution to this problem (see also chapter 4 on page 28). Design patterns often seem to be very similar or they can be described in similar ways, however, their small differences are quite important. Patterns are sometimes based on related concepts which is the reason why their deeper meaning can only be caught by studying their application and its resulted consequences. As carefully worded definitions of the patterns can be found in the design pattern book [GHRV95]. In this document only a few patterns are used, the most important one in this scope will be discussed in the next section. It is a basic pattern that is often used although it is not explicitly mentioned. Further readings about Design Patterns is well documented in [Ale01] and [Sut00]. 4.3.1 Hook and HookUp The Hook Hook-Up pattern (refer to [Sch02]) is split up in two parts, the Hook and the HookUp (see following subsections). This pattern is often used unconsciously by other patterns like the Observer pattern (see [GHRV95]). The deeper intention of the Hook / Hook-Up pattern is to connect components. Moreover, it is not described in detail in the Design Patterns Book [GHRV95] because it was later defined by Klaus Schmaranz, [Sch02]. Some patterns use the hook pattern for connecting components, but depending on the behavior of the connected components these patterns describe different func- tionalities. For example, the Observer pattern combines components by using the hook pattern but in contrast to a Mediator the observer notifies all connected components when an event occurred. The mediator that also uses the hook mechanism for connecting the components, chooses only one component to forward the event.
  56. 56. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 41 There are some other patterns that use the hook mechanism, but it is not the aim of this document to explain all design patterns, however, for a better understanding of the next sections it is useful, to have a closer look on the hook pattern. Hook The definition of a Hook consists of two parts: the abstract Hook and the concrete Hook. The abstract Hook defines an interface for a component that represents me- thods (e.g. execute) how this component can be used. A concrete Hook implements this abstract Hook interface. The Hook-method is the “reason” why a class is written. It contains the functiona- lity that is needed. If more than one Hook method is needed, several abstract Hook interfaces must be defined, to make clear how this component can be used. In the service-example (section A.2 on page 88) the Service interface is an abstract Hook-Up and the Echo class is a concrete hook because of the execute method that is used to ask the service. Hook-Up The Hook-Up is the second part of the Hook Hook-Up pattern. In comparison with the Hook that defines what happens when a method is called, the Hook-Up decides when the method is invoked. The Hook-Up can also be split up into two parts: the abstract Hook-Up and the concrete Hook-Up. An abstract Hook-Up defines a method that is used to connect a Hook with another component. A concrete Hook-Up is a component that imple- ments the abstract Hook-Up interface. In the example the ServiceProvider (Listing A.2 on page 92) the interface is an abstract Hook-Up because it contains the Hook-Up method register(Service). The ServiceProviderImpl class is a concrete Hook-up.
  57. 57. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 42 4.3.2 Components The idea of components is based on the main concept of object-oriented program- ming. [Sch02] • A component is responsible for one function or can deal with one type of data. • A component must be free of side-effects. • The communication with a component is only allowed through methods, public member-variables are not admitted. • There are only a few ways how this class can be used. Because of these constraints components are able to signal missusage. Mis-usage can only occur by applying wrong values to methods or by calling methods in a wrong sequence. Developers often violate these ideas which are also an important part of the OOP approach, so the stronger term component was defined to make clear, that if someone is writing a component, this class complies with these ideas and limitations. In the service example the different services are typical components. Some guidelines how components should be designed will now be discussed. These guidelines widely correlate with guidelines for designing exchangeable and expand- able components, and they also force the design of good testable applications. So complying with them, these guidelines are not a restriction to the freedom of software-designers but rather implicate a good design for an exchangeable and ex- pandable component-framework. Side-effect-free : A component must avoid side effects. It should encapsulate the functionality without affecting other components (e.g. changing global variables), except those components that are registered in order to receive reactions from the component.
  58. 58. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 43 Composition by Hook and Hook-Up : The Hook Hook-Up pattern must be used for binding components [Sch, Chap. 3.4.7]. This includes the requirement for defined interfaces for each Hook and Hook-Up. The interfaces describe how a component is registered and which methods can be used. The Hook Hook-Up pattern allows test teams to connect components via a proxy that can be used for monitoring the test. This can be done dynamically without recompiling the component. Dynamically loadable : This correlates with the guideline of using the Hook Hook-Up pattern. Components must be dynamically loadable in order to replace them without recompiling the framework. Moreover, this allows the usage of components that are not known at compile-time. [Mey97, Chap. 2] Thinking of the inner state of a component : During the design process the inner state of a component should always be present. The inner state of a component will be changed by calling a method. If the method finishes correctly, the next state is reached. If an error occurs during the execution, another state is reached. Thinking of the effects of method-calls will help to design components with small interfaces. Usage of exceptions : Exceptions have to be triggered on unpredictable events or mis-usages of a component. Even wrong method-calls should trigger excep- tions because a programmer is not forced to handle special return-values. If a method returns true when it succeeds and false when it fails, the programmer can ignore this result. If an exception is thrown when it fails, the program- mer must handle it. Using exceptions will also force designers to think about the handling of missusage or wrong method calls during the design. Thinking about the impact of an exception on the inner state of a component during the design-phase facilitates a robust component design. Notice that it is not allowed to use exceptions for controlling program flow because, this would vi- olate the concept of an exception. Exceptions are not designed to handle the default cases but to indicate dangerous situations (see section 4.4 on page 44).
  59. 59. CHAPTER 4. GENERAL SOFTWARE QUALITY FACTORS 44 Definition of the usage : Components must be defined with respect to the con- text, where they are used. Therefore, obligations and benefits with reference to the context can be defined. This definition can also be called contract. For more details about contracts refer to [Val03, chapter 4, p.34]. 4.4 Exceptions Even when all components are correctly used difficulties can occur, e.g. an inter- rupted network connection. Program should be able to manage these situations. Exceptions are the correct tool to solve such problems. An exception can be interpreted as a possible result of a method call that occurs only when something goes wrong. This is an important aspect because exceptions are thrown when a dangerous situation has been reached, whereas they must not be used for controlling the program flow. If the program flow depends on a result of a method call, the result value should be used and not a possible thrown exception. Several languages implement the exception-concept in different ways. Java, for example supports exceptions as a part of the language concept by providing special keywords to handle these situations. All implementations allow the definition of an additional program code that is executed when a dangerous situation has been signaled by throwing an exception. Notice that for the test design an exception can be a valid test result. It is necessary to test if an exception will occur caused by missusage or an unexpected event in the application. The emergence of an exception is part of the component behavior.
  60. 60. CHAPTER 5. TESTING 45 Chapter 5 Testing The English Positivist philosophers Locke, Berkley, and Hume said that anything that can’t be measured doesn’t exist. When it comes to code, I agree with them completely, Software features that can’t be demonstrated by automated tests simply don’t exist. Kent Beck Another philosophic approach to testing can be found by interpreting articles by Sir Karl Popper. He defines the matter of a scientific thesis or a scientific work by two factors, the logical and the empirical matter [Pop94, S 40]. The logical matter combines the sum of all works that can be derived from a thesis. This can be interpreted as the reusablility of a component. On the other hand, the empirical matter of a work is defined by the sum of all proves that demonstrate the correctness of a thesis. This second factor can be interpreted as the testability of a component. During the last few years patterns and components have become popular so more and more papers or books about these “solutions” can be found in nearly every software design division. As a consequence we claim that the evolution of writing reusable and expendable software components by using the ideas of the design patterns is nearly finished. We can say that the IT community has learned to carry out the
  61. 61. CHAPTER 5. TESTING 46 ideas of reusable and extendable components and is able to fulfill the goals of the OOP approach. So the logical matter of components that define an application can now be satisfied. But what about the second factor? The aim of this section is to discuss different test concepts. It will also show how the component-based approach which has been described in earlier sections, allows to define tests and thus it facilitates to increase the empirical matter. How can this be done and what are the steps to achieve this goal? 5.1 Goals for running tests In this scope another aspect become more important: Do these reusable components react as it has been defined during the design? At this point new terms gain impor- tance correctness and robustness. The target of a test is to check implementations against these terms. If someone is in the situation that the test of a new software component must be defined, there are two frequently asked questions: • What are the goals of this test? • Does this test verify the correctness of a component? However, these questions are dangerous because they imply that a test is carried to show the correctness of a program, which is wrong! A test is done in order to find failures or misbehaviors of a program and not to show that everything functions correctly. This psychological aspect of testing is one of the reasons why testing is not as popular as designing. Testing is a destructive task that charges the components in order to affect them to cause errors. It cannot be done by using the default test-cases,as they will result in default test results. Instead it must be done by using test-cases that are more complex than the default ones and
  62. 62. CHAPTER 5. TESTING 47 this must be carried out in an environment that is close to the real-life situations in which the component will be used. A typical situation is that after a test someone asks: “...ok, this was the test, but did it fail or succeed?” Such situations must be avoided! It must be clear how a result of a test run has to be interpreted before the test is done. It is therefore helpful to use tools that are able to generate clear summaries of what has happened during the test. At least almost all scenarios that are discussed in this work deal with the problem of verifying the correctness and robustness of a program, but there are other possible targets too. 5.1.1 Targets of test runs Keeping the main goal of finding errors in mind other useful targets can be defined for the test. For all targets it must be clear that a test is done to find misbehaviors and not to show that “everything is ok” in order to ease the development team’s conscience. It is not the aim of this work to give a complete introduction into all possible test scenarios, but it is necessary to keep all requirements of a software project in mind when the tests are designed. It is likely that not all requirements of a project can be proved by only showing the correctness and robustness of a program and therefore testing this aspects is an important but not sufficient part of the whole test process. What are possible goals for test runs? Correctness-tests : Testing the correctness of an application or a component can only be done when its behavior is known and well-documented. The correctness of a component can be tested by applying method calls in the correct order on the component. These calls must also use correct parameters so that the component is used correctly. If there are different ways how the component can be used, all ways must be included in the test. For example, if a class can be instantiated by using different constructors, all these constructors

×