Your SlideShare is downloading. ×
Maturing software engineering knowledge through classifications
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Maturing software engineering knowledge through classifications


Published on

Published in: Education, Technology

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. A warm Good Evening!!! 1
  • 2. IEEE Transactions on Software Engineering, July/August 2009 Suraj Kumar Bhandari MTech in IT Kathmandu University Nepal 2
  • 3. ContentsIntroductionBackground and MotivationRelated work Research Objective Literature ReviewCalculating Congruence Research Methodology Results Conclusions and further work Your view 3
  • 4. Introduction In Science and Engineering, knowledge matures as the investigates objects areclassified. Classification groups similar objects to form an organization. Mature knowledge is not a sequential heap of pieces of knowledge but anorganized structure of knowledge items, where each piece smoothly andelegantly fits into place as in a puzzle. Classification has advanced knowledge in three ways:1. By providing a set of unifying constructs2. By understanding interrelationships3. By identifying knowledge gaps However, classifications can serve other purposes apart from providing a useful organization of knowledge. In case of engineering the purpose is usually decision making support. 4
  • 5. Introduction contd… Classifications structure a body of knowledge with in a field, enablingresearchers and practitioners to generalize, communicate, and apply the findings. The information organized as a classification is a step higher on the knowledgematurity ladder than separate pieces of information, as it can be more easilyunderstood and more efficiently retrieved. Due to breadth of Software Engineering knowledge, any attempt at classifying allknowledge would be an immense undertaking for just one research group. The testing area can benefit from testing techniques classification. Today, there is multitude and diversity of items to be classified, and testingtechniques meet the conditions that illustrate the difficulties and benefits ofclassification in Software Engineering. They propose an initial classification of testing techniques using a set of the unittesting techniques available today. Although it is directly applicable to testingtechniques, the approach they have followed in this research can be applied toother classification of Software engineering as well. 5
  • 6. Background and motivation A number of different objects have already bee classified in the field of SoftwareEngineering. They were:- Bass et al’s classifications of architectural styles , this classification was designed to unify software system architecture descriptions to provide developers with information about particular style and offer guidance for selecting the styles that are to dictate the design of a software system. Maiden and Rugg, they presented a classification of requirements elicitation methods to ease the selection of methods and help developers set up and acquisition program. Glass et al., Ramesh et al., Vessey et al., they outlines a classification of the state of research in the field of Computer Science, Software Engineering and Information Systems. Chillarege, he describes a classification of defects aimed at rapidly capturing the semantics of each software defect. Finally, SWEBOK, it is a classification of knowledge now available in the field of Software Engineering. The SWEBOK was designed to provide a consensually validated chacterization of the bounds of the discipline of topical access to the underlying body of knowledge. There were others classifications techniques but they were not built for the same purpose as like the unit testing techniques. 6
  • 7. Related workThree testing techniques classifications have been proposed to date:1. Bertilino’s classifications of testing techniques2. Glass’ classification of testing techniques3. Zhu et al’s classification of testing techniques1. Bertilino classifications of testing techniques:- It considers just one criterion It accounts for how the techniques generates test cases and has six possible values: - 1. Tester’s insight and classification 2. Specification 3. Code structure 4. Faults to be discovered 5. Field usage and 6. Application type 7
  • 8. Related work cotd…. 2. Glass’ classifications of testing techniques:-  It also considers a single criterion.  It is related to the goal of the techniques ( testing all requirements, testing the structure of the program, etc.)  It takes four different values: 1. Requirements- driven 2. Structure- driven 3. Statistics- driven 4. Risk- driven 3. Zhu et al’s classifications of testing techniques:-  It classifies the techniques according to two criteria: 1. It is based on source of the information used to generate test cases ( based on Glass’ goal)  It can take four different values ( which can be traced back to the traditional black/ white box classes) 1. Specifications 2. Interface 3. Program 8 4. Program and its specifications.
  • 9. Related work cotd….2. It is based on the underlying testing approach(similar to how Bertolino’s testsare generated)  It takes three possible values: 1. Code Structure 2. Program faults 3. Programmers errors However though these classification agrees on some criteria, their value aredifferent. This leads to different testing technique classifications. Additionally, Bertilino’s and Zhu et al’s classifications list types of testingtechniques, not actual testing techniques.Lastly, these classifications are valuable contributions to the testing field as theyprovided the groundwork for testing techniques research. 9
  • 10. Fig 1:- Bertolino’s Classifications of Testing Techniques 10
  • 11. Research Objective1. Identify the underlying classification criteria.2. Search and analyze the sources of information required to give value to the classification criteria for each technique.3. Instantiate and accordingly order the classification criteria for the techniques that are being classified.4. Find out the best appropriate method for unit testing technique. 11
  • 12. Literature Review What is unit testing ?  Unit testing is a method by which individual units of source code are tested to determine if they are fit for use. A unit is the smallest testable part of an application. In procedural programming, a unit could be an entire module but is more commonly an individual function or procedure. In object-oriented programming , a unit is often an entire interface, such as a class, but could be an individual method. Unit tests are created by programmers or occasionally by white box testers during the development process. Unit testing is commonly automated, but may still be performed manually. The IEEE does not favor one over the other. A manual approach to unit testing may employ a step-by-step instructional document. Nevertheless, the objective in unit testing is to isolate a unit and validate its correctness. Automation is efficient for achieving this, and enables the many benefits listed in this article. 12
  • 13. Literature Review cotd… They found that the classification scheme for unit testing is iterative and generic. Soto make their scheme easier to understand, they arranged the criteria around twoperspectives related to the type of information they presented.1. Operational :- It links how the technique works. The operational group were grouped around five aspects : 1. Technique 2. Test Cases 3. Object 4. Tools 5. Agents. From these five aspects 24 classification criteria identified.2. Historical:- It refers to the experience in using the techniques. The historical groupwere grouped around 2 aspects:1. Project2. Satisfaction From these two aspects 6 classification criteria identified. So total of 30 classification criteria were identified form both group. Out of those 30, they 13 testing techniques rather than exhaustive classification. 13
  • 14. Literature Review cotd… Problems Identified:- 1. Values were unreliable. In textbook and research articles, they came across inconsistent information on the value of some techniques for criteria were like dependencies, repeatability, type of defects and the effectiveness. 2. Values were unknown. So the completeness and precision criteria cannot be rated at present. 3. Metrics for assigning value to some criteria are not equivalent. This means that information provide different metrics from the ones applied in theirs classifications. 4. Some values were not publicly available. The information contained in the historical perspective represents knowledge gained by organization after using the techniques in real projects. Since most organization do not now document this information, it is difficult to find. 14
  • 15. Literature Review cotd…To try out the proposal they presented a testimonial classification of a wide- ranging13 technique techniques. The figure below explains the 13 testing techniques:- 15
  • 16. Calculating Congruence:-1. A detailed systematic characterization of testing techniques2. Understand the testing technique3. Analysis of knowledge gaps4. Classification use for selection purpose 1. A detailed systematic characterization of testing techniques:-  They analyze the equivalences between the white/black box, Bertolino’s, Glass’ and Zhu et al.’s classification and other propose in this experiment.  Bertolino’s and Zhu et al.’s classification share the test case generation criterion/criteria.  Bertolino’s classification based on tester’s insight and experience value and Zhu et al.’s error-based value are related to the knowledge and experience criteria of this classifications. 16
  • 17. Calculating Congruence contd…  Zhu et al.’s , glass and the white/black box classification share the source of information criteria.  Their new classification suggests that techniques differ not only as regards the input criteria but also as to the test data cost and the size criteria. Finally, there aim was to produce a selective classification i.e. ending in a single technique. 2. Understanding testing technique:-  To understand the testing technique, the grouped five different testing technique groups by investigating the proximity and distance between the techniques. Group 1:- All c-uses, all p-uses, and all du-paths. It is formed by techniques whose values are code for inputs, high for test data cost, Medium for size, yes for availability, No for support, Low for comprehensibility. 17
  • 18. Calculating Congruence contd…Fig:- Groups of testing technique 18
  • 19. Calculating Congruence contd…  Group 2:- Mutation and selective mutation . It is formed by techniques whose values are code for inputs, high for test data cost, medium for size, yes for availability, No for support, and high for comprehensibility.  Group 3:- Statement testing, branch testing, path testing. It is formed by techniques whose values Code for inputs, High for test data cost, Medium for size, yes for availability, Yes for support and High for comprehensibility. Group 4:- Thread testing and all possible rendezvous. It is formed by techniques whose values Code for inputs, High for test data cost, Medium for size, no for availability, No for support and medium for comprehensibility. Group 5:- Random testing and boundary value analysis. It is formed by techniques whose values specification for inputs, low for test data cost, for size, no for availability, No for support and high for comprehensibility. 19
  • 20. Calculating Congruence contd… 3. Analysis of knowledge gaps:-  Software Engineering knowledge classification is useful for detecting knowledge gaps.  For every real gap detected, they identify the missing item.  First of all, they expand the classification tree by considering all the possible value for each classification criteria.  Each tree node represents a possible value for classification criteria at the respective level, irrespective of whether or not there are techniques with this value.  Each tree node with no associated technique at the end of a path is a gap.  The gap is circled in the classification is shown in the table in next page:- 20
  • 21. Calculating Congruence contd… Fig:- Expanded classification of testing technique 21
  • 22. Calculating Congruence contd… 4. Classification use for selection process:-  A key goal of engineering classification is to support decision making.  Practitioners can use a decision tree for the selection purpose.  The most restrictive criteria have been placed at the top of the tree.  The features of the software object to be tested appear at the top, followed by the characteristics of the available testers then the properties of the test cases to be generated, available tools, and the actual technique.  Finally, a decision tree for selecting testing techniques is developed with omitting the knowledge gaps. We will see that in RESULTS section. 22
  • 23. Research MethodologyResearch Categories:-1. Quantitative Data Analysis:- Theoretical and empirical research based on n extensive Literature Review, testing and empirical journals and conferences. Example:- Survey1. Qualitative Data Analysis:- Aim to find out the appropriate unit testing techniques. The qualitative method investigates the why and how of decision making not just what, where, when. Example :- Interviews, observation etc. 23
  • 24. Research Methodology contd…Data collection technique:- Data Sources:- From Journals, conferences they identified 69 relevant papers corresponding to 47 studies.  13 Theoretical  32 Empirical  2 Simulation  SWEBOK Questionnaires:-  5 set of questionnaires for interview and sample size Sample Size:-  Citizens : 17 ( 3 from universities, 2 from university-associated research centers and 12 from software companies holding different positions:- 3 professors, 5 software development managers, 2 project managers, six developers and one scientist ). 24
  • 25. Research MethodologyQualitative Data Analysis:- Analysis of structured questionnaire interview and research paper of :- Unit Testing Experts Final year students of Computing of the Universidad Politecnica de Madrid  Software Development Managers  Professors  Scientist  Project Managers  Developers etc. 25
  • 26. Results1. Classification Efficiency :- The experiment showed that the learning and selection time is shorter using the classification and the time spent consulting the classification can be considered negligible against the learning and selection time.2. Classification Usability:- The subjects had fewer problems using the classification and the problem frequency is lower with the classification.3. Classification Completeness:- The main finding was that it should include all the criteria that originally appeared in the classification scheme for it to be useful to practitioners.4. User Satisfaction:- The subjects can be said to like the classification and would be prepared to use it if given the opportunity.s 26
  • 27. ResultsFig: Decision tree for selecting test technique 27
  • 28. Conclusion and Further Work  Classification serve three purposes:- 1. Identify the classification criteria 2. Analyze the information needed to classify the objects 3. Instantiate the set of classification criteria for each object and order the set  This process has been proven to be useful for putting together a classification that covers and enriches unit testing technique.  They found that the classification presented here gives a plausible explanation of the success of testing technique today. They also examine the gap there were in the testing technique. Finally they showed that how to use the classification for selection process. Although the proposed classification improves selection, the tester still responsible for decision making. 28
  • 29. Your View?? 29
  • 30. Thank You!!! 30