Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A Regression Analysis Approach for Building a Prediction Model for System Testing Defects


Published on

Paper presented during Postgraduate Research Workshop at Software Testing Conference 2012 (SOFTEC2012)

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

A Regression Analysis Approach for Building a Prediction Model for System Testing Defects

  1. 1. A Regression Analysis Approach for Building a Prediction Model for System Testing Defects (Paper No: S1-3) Muhammad Dhiauddin bin Mohamed Suffian Faculty of Computer Science & Information System AP Dr. Suhaimi Ibrahim Advanced Informatics School
  2. 2. Presentation Outline • • • • • Introduction Related Works Research Methodology Findings and Discussion Conclusion and Recommendation
  3. 3. Introduction • Defect prediction is very significant to the independent testing team – ensure all potential field defects could be successfully contained within system testing phase – defects could be prevented from escaping to the end-users – achieve the target of zero known post release defects for the software delivered to end-users • Common understanding of defect  forecast defects in software [1][2] • Defect prediction for system testing – predict failures in system testing instead of defects [3] – predict remaining defects in software release as part of test process simulation [4]
  4. 4. Introduction (cont.) • Motivation to undertake this research effort: Assigning appropriate number of test engineers across multiple test projects • Re-align test execution to meet deadline • Action plan when actual ≠ prediction • Right test scenarios to capture predicted defects • Better root cause analysis • Decision by management on software release • Stability of whole development process
  5. 5. Introduction (cont.) • Objectives: – To analyze existing techniques of building prediction model for system testing defects – To build prediction model for system testing defects using statistical approach – To evaluate the proposed prediction model based on specified acceptance criteria.
  6. 6. Related Works • Defect terminology: – any flaw in the system or even in the system’s components could cause the system to malfunction [5] – deviation from its specification: physical software or work products [6] – imperfection in the process as well as work product besides software [7] – defect generated from V & V activities
  7. 7. Related Works (cont.) • Approaches to defect prediction: – Term is used interchangeably with defect estimation to describe the proactive process of characterizing defects found in software in producing high quality product [8] – size and complexity metrics: McCabe’s cyclomatic complexity as well as lines of code (LOC) e.g. Defect = 4.86 + 0.018 Lines of Code [6] – categorized into project management, work product assessment and process improvement [7] – used Rayleigh Model to predict defect density at different phases of project life cycle ]9] – combination of product and project metrics via regression analysis [10] – used mathematical distributions as quality prediction model as part of software fault prediction techniques [11] – Used development information as important factor for the prediction and model quality [12] – applying object-oriented metrics for predicting faults in open source software [13] – several inputs can be used to simulate system test phase in SDLC [14] – applied statistical method in Six Sigma to predict defect density [15] – used defect decay model to predict remaining defects in on-going testing process [16]
  8. 8. Related Works (cont.) • Issues: – Strength: Easy to use, efficient, effective and able to indicate the process performances; Weakness: need to have sampling, require stable process and does not account for changes [7] – Critiques [6]: • • • • • unknown relationship between defect and failures problems with multivariate statistical approach, problems of using size and complexity metrics as sole predictors of defects problems in statistical methodology and data quality false claims about software decomposition • Measuring the success: – measuring the percent of faults found in the identified files [17] – help in maintenance resource planning as well as software insurance [18]
  9. 9. Research Methodology Source of data: ONE applied R&D organization V-shaped process model Metrics from V&V: •requirement review •design review •test plan review •test cases review •code inspection & unit testing •system testing Software type: •Web-based •Componentbased Language: •PHP •.NET •Java
  10. 10. Findings and Discussion Prediction of System Testing Defects
  11. 11. Findings and Discussion(cont.) Initial data set Initial regression analysis result
  12. 12. Findings and Discussion (cont.) Revised data set
  13. 13. Findings and Discussion (cont.)
  14. 14. Findings and Discussion (cont.) Verification result Selected prediction model for initial implementation Functional Defects – = 4.00 - 0.204 Requirement Error - 0.631 Coding Error + 1.90 KLOC – 0.140 Requirement Page + 0.125 Design Page 0.169 Total Test Cases + 0.221Total Effort Days
  15. 15. Conclusion and Recommendation • Achievement: – Demonstrated the successful construction of prediction model for system testing defects by applying regression analysis approach – Demonstrated the ability to predict defects for system testing by using metrics in requirement, design and coding phase • Future works: – To predict non-functional defects such as performance, security and usability defects – To predict defects based on severity of defects i.e. critical, major and minor defects – To incorporate more factors in building similar model such as function point, programming languages, and number of classes – To develop software tool that could dynamically generate the latest prediction equation in real time and assist in prediction activity
  16. 16. THANK YOU