The document provides an overview of research conducted on classifying unit testing techniques. It discusses previous work on testing technique classifications and identifies their limitations. The research aims to develop a new classification scheme using operational and historical criteria. 30 classification criteria are identified and used to analyze 13 testing techniques. The results show the new classification improves efficiency and usability when selecting techniques compared to previous schemes. Knowledge gaps in existing techniques are also examined.

1. A warm Good Evening!!!
1

2. IEEE Transactions on Software Engineering, July/August 2009
Suraj Kumar Bhandari
MTech in IT
Kathmandu University
Nepal
2

3. Contents
Introduction
Background and Motivation
Related work
 Research Objective
 Literature Review
Calculating Congruence
 Research Methodology
 Results
 Conclusions and further work
 Your view
3

4. Introduction
 In Science and Engineering, knowledge matures as the investigates objects are
classified.
 Classification groups similar objects to form an organization.
 Mature knowledge is not a sequential heap of pieces of knowledge but an
organized structure of knowledge items, where each piece smoothly and
elegantly fits into place as in a puzzle.
 Classification has advanced knowledge in three ways:
1. By providing a set of unifying constructs
2. By understanding interrelationships
3. By identifying knowledge gaps
 However, classifications can serve other purposes apart from providing a
useful organization of knowledge. In case of engineering the purpose is
usually decision making support.
4

5. Introduction contd…
 Classifications structure a body of knowledge with in a field, enabling
researchers and practitioners to generalize, communicate, and apply the findings.
 The information organized as a classification is a step higher on the knowledge
maturity ladder than separate pieces of information, as it can be more easily
understood and more efficiently retrieved.
 Due to breadth of Software Engineering knowledge, any attempt at classifying all
knowledge would be an immense undertaking for just one research group.
 The testing area can benefit from testing techniques classification.
 Today, there is multitude and diversity of items to be classified, and testing
techniques meet the conditions that illustrate the difficulties and benefits of
classification in Software Engineering.
 They propose an initial classification of testing techniques using a set of the unit
testing techniques available today. Although it is directly applicable to testing
techniques, the approach they have followed in this research can be applied to
other classification of Software engineering as well.
5

6. Background and motivation
 A number of different objects have already bee classified in the field of Software
Engineering. They were:-
 Bass et al’s classifications of architectural styles , this classification was designed
to unify software system architecture descriptions to provide developers with
information about particular style and offer guidance for selecting the styles that
are to dictate the design of a software system.
 Maiden and Rugg, they presented a classification of requirements elicitation
methods to ease the selection of methods and help developers set up and
acquisition program.
 Glass et al., Ramesh et al., Vessey et al., they outlines a classification of the state of
research in the field of Computer Science, Software Engineering and Information
Systems.
 Chillarege, he describes a classification of defects aimed at rapidly capturing the
semantics of each software defect.
 Finally, SWEBOK, it is a classification of knowledge now available in the field of
Software Engineering. The SWEBOK was designed to provide a consensually
validated chacterization of the bounds of the discipline of topical access to the
underlying body of knowledge.
 There were others classifications techniques but they were not built for the same
purpose as like the unit testing techniques. 6

7. Related work
Three testing techniques classifications have been proposed to date:
1. Bertilino’s classifications of testing techniques
2. Glass’ classification of testing techniques
3. Zhu et al’s classification of testing techniques
1. Bertilino classifications of testing techniques:-
 It considers just one criterion
 It accounts for how the techniques generates test cases and has six possible
values: -
1. Tester’s insight and classification
2. Specification
3. Code structure
4. Faults to be discovered
5. Field usage and
6. Application type
7

8. Related work cotd….
2. Glass’ classifications of testing techniques:-
 It also considers a single criterion.
 It is related to the goal of the techniques ( testing all requirements, testing the
structure of the program, etc.)
 It takes four different values:
1. Requirements- driven
2. Structure- driven
3. Statistics- driven
4. Risk- driven
3. Zhu et al’s classifications of testing techniques:-
 It classifies the techniques according to two criteria:
1. It is based on source of the information used to generate test cases ( based
on Glass’ goal)
 It can take four different values ( which can be traced back to the traditional
black/ white box classes)
1. Specifications
2. Interface
3. Program
8
4. Program and its specifications.

9. Related work cotd….
2. It is based on the underlying testing approach(similar to how Bertolino’s tests
are generated)
 It takes three possible values:
1. Code Structure
2. Program faults
3. Programmers errors
 However though these classification agrees on some criteria, their value are
different. This leads to different testing technique classifications.
 Additionally, Bertilino’s and Zhu et al’s classifications list types of testing
techniques, not actual testing techniques.
Lastly, these classifications are valuable contributions to the testing field as they
provided the groundwork for testing techniques research.
9

10. Fig 1:- Bertolino’s Classifications of Testing Techniques
10

11. Research Objective
1. Identify the underlying classification criteria.
2. Search and analyze the sources of information required to give value to
the classification criteria for each technique.
3. Instantiate and accordingly order the classification criteria for the
techniques that are being classified.
4. Find out the best appropriate method for unit testing technique.
11

12. Literature Review
What is unit testing ?
 Unit testing is a method by which individual units of source code are tested to
determine if they are fit for use.
A unit is the smallest testable part of an application. In procedural programming, a
unit could be an entire module but is more commonly an individual function or
procedure. In object-oriented programming , a unit is often an entire interface, such
as a class, but could be an individual method. Unit tests are created by
programmers or occasionally by white box testers during the development process.
Unit testing is commonly automated, but may still be performed manually.
The IEEE does not favor one over the other. A manual approach to unit testing may
employ a step-by-step instructional document. Nevertheless, the objective in unit
testing is to isolate a unit and validate its correctness. Automation is efficient for
achieving this, and enables the many benefits listed in this article.
12

13. Literature Review cotd…
 They found that the classification scheme for unit testing is iterative and generic. So
to make their scheme easier to understand, they arranged the criteria around two
perspectives related to the type of information they presented.
1. Operational :- It links how the technique works. The operational group were
grouped around five aspects :
1. Technique
2. Test Cases
3. Object
4. Tools
5. Agents.
 From these five aspects 24 classification criteria identified.
2. Historical:- It refers to the experience in using the techniques. The historical group
were grouped around 2 aspects:
1. Project
2. Satisfaction
 From these two aspects 6 classification criteria identified. So total of 30
classification criteria were identified form both group. Out of those 30, they 13
testing techniques rather than exhaustive classification.
13

14. Literature Review cotd…
Problems Identified:-
1. Values were unreliable. In textbook and research articles, they came
across inconsistent information on the value of some techniques for
criteria were like dependencies, repeatability, type of defects and the
effectiveness.
2. Values were unknown. So the completeness and precision criteria
cannot be rated at present.
3. Metrics for assigning value to some criteria are not equivalent. This
means that information provide different metrics from the ones
applied in theirs classifications.
4. Some values were not publicly available. The information contained in
the historical perspective represents knowledge gained by
organization after using the techniques in real projects. Since most
organization do not now document this information, it is difficult to
find.
14

15. Literature Review cotd…
To try out the proposal they presented a testimonial classification of a wide- ranging
13 technique techniques. The figure below explains the 13 testing techniques:-
15

16. Calculating Congruence:-
1. A detailed systematic characterization of testing techniques
2. Understand the testing technique
3. Analysis of knowledge gaps
4. Classification use for selection purpose
1. A detailed systematic characterization of testing techniques:-
 They analyze the equivalences between the white/black box,
Bertolino’s, Glass’ and Zhu et al.’s classification and other propose
in this experiment.
 Bertolino’s and Zhu et al.’s classification share the test case
generation criterion/criteria.
 Bertolino’s classification based on tester’s insight and experience
value and Zhu et al.’s error-based value are related to the
knowledge and experience criteria of this classifications.
16

17. Calculating Congruence contd…
 Zhu et al.’s , glass and the white/black box classification share the source of
information criteria.
 Their new classification suggests that techniques differ not only as regards the
input criteria but also as to the test data cost and the size criteria.
Finally, there aim was to produce a selective classification i.e. ending in a single
technique.
2. Understanding testing technique:-
 To understand the testing technique, the grouped five different testing technique
groups by investigating the proximity and distance between the techniques.
Group 1:- All c-uses, all p-uses, and all du-paths. It is formed by techniques
whose values are code for inputs, high for test data cost, Medium for size, yes
for availability, No for support, Low for comprehensibility.
17

18. Calculating Congruence contd…
Fig:- Groups of testing technique 18

19. Calculating Congruence contd…
 Group 2:- Mutation and selective mutation . It is formed by techniques whose
values are code for inputs, high for test data cost, medium for size, yes for
availability, No for support, and high for comprehensibility.
 Group 3:- Statement testing, branch testing, path testing. It is formed by
techniques whose values Code for inputs, High for test data cost, Medium for
size, yes for availability, Yes for support and High for comprehensibility.
Group 4:- Thread testing and all possible rendezvous. It is formed by techniques
whose values Code for inputs, High for test data cost, Medium for size, no for
availability, No for support and medium for comprehensibility.
Group 5:- Random testing and boundary value analysis. It is formed by
techniques whose values specification for inputs, low for test data cost, for size,
no for availability, No for support and high for comprehensibility.
19

20. Calculating Congruence contd…
3. Analysis of knowledge gaps:-
 Software Engineering knowledge classification is useful for detecting knowledge
gaps.
 For every real gap detected, they identify the missing item.
 First of all, they expand the classification tree by considering all the possible
value for each classification criteria.
 Each tree node represents a possible value for classification criteria at the
respective level, irrespective of whether or not there are techniques with this
value.
 Each tree node with no associated technique at the end of a path is a gap.
 The gap is circled in the classification is shown in the table in next page:-
20

21. Calculating Congruence contd…
Fig:- Expanded classification of testing technique 21

22. Calculating Congruence contd…
4. Classification use for selection process:-
 A key goal of engineering classification is to support decision making.
 Practitioners can use a decision tree for the selection purpose.
 The most restrictive criteria have been placed at the top of the tree.
 The features of the software object to be tested appear at the top, followed by
the characteristics of the available testers then the properties of the test cases to
be generated, available tools, and the actual technique.
 Finally, a decision tree for selecting testing techniques is developed with
omitting the knowledge gaps. We will see that in RESULTS section.
22

23. Research Methodology
Research Categories:-
1. Quantitative Data Analysis:- Theoretical and empirical research based on
n extensive Literature Review, testing and empirical journals and
conferences. Example:- Survey
1. Qualitative Data Analysis:- Aim to find out the appropriate unit testing
techniques. The qualitative method investigates the why and how of decision
making not just what, where, when. Example :- Interviews, observation etc.
23

24. Research Methodology contd…
Data collection technique:-
Data Sources:- From Journals, conferences they identified 69 relevant
papers corresponding to 47 studies.
 13 Theoretical
 32 Empirical
 2 Simulation
 SWEBOK
Questionnaires:-
 5 set of questionnaires for interview and sample size
Sample Size:-
 Citizens : 17 ( 3 from universities, 2 from university-associated
research centers and 12 from software companies holding different
positions:- 3 professors, 5 software development managers, 2
project managers, six developers and one scientist ).
24

25. Research Methodology
Qualitative Data Analysis:-
 Analysis of structured questionnaire interview and research paper of :-
Unit Testing Experts
Final year students of Computing of the Universidad Politecnica de
Madrid
 Software Development Managers
 Professors
 Scientist
 Project Managers
 Developers etc.
25

26. Results
1. Classification Efficiency :- The experiment showed that the learning and
selection time is shorter using the classification and the time spent consulting
the classification can be considered negligible against the learning and
selection time.
2. Classification Usability:- The subjects had fewer problems using the
classification and the problem frequency is lower with the classification.
3. Classification Completeness:- The main finding was that it should include all
the criteria that originally appeared in the classification scheme for it to be
useful to practitioners.
4. User Satisfaction:- The subjects can be said to like the classification and
would be prepared to use it if given the opportunity.s
26

27. Results
Fig: Decision tree for selecting test technique 27

28. Conclusion and Further Work
 Classification serve three purposes:-
1. Identify the classification criteria
2. Analyze the information needed to classify the objects
3. Instantiate the set of classification criteria for each object and order
the set
 This process has been proven to be useful for putting together a
classification that covers and enriches unit testing technique.
 They found that the classification presented here gives a plausible
explanation of the success of testing technique today.
They also examine the gap there were in the testing technique.
Finally they showed that how to use the classification for selection
process.
Although the proposed classification improves selection, the tester still
responsible for decision making.
28

29. Your View??
29

30. Thank You!!!
30