Second Progress Presentation
                         On
    Test Effort Estimation In Regression Testing




Presented By:                          Supervisor:
Abhilasha,                             Mr. Ashish Sharma,
M. Tech,                               Department of CEA,
Enroll :118150001.                     GLA university.
Outline

      Problem Statement
      Literature review
      Critical Analysis
      Conclusion
      References




     12/14/2012            2
Problem Statement:



          “Test Effort Estimation In Regression Testing”




  12/14/2012                                               3
Literature review

     The research work is formulated in two modules :

                    -Test Effort Estimation
                    -Regression Testing




  12/14/2012                                            4
Module I :Test Effort Estimation

   • Test effort estimation is the estimation of the testing size, testing cost and
     testing schedule for a specified software test project by defined methods,
     tools and techniques[1].

   • The test effort is defined as the sum of the effort spent on test activity and
     the effort spent on debug activity [2].




   12/14/2012                                                                         5
Literature Review
 Paper#1
 Title: Test Effort Estimation Models Based On Test Specifications

 Author: Eduardo Aranha,and Paulo Borba, IEEE Testing: Academic and Industrial
         Conference - Practice And Research Technique,pp.67-71,2007.

                                        Proposal


     In this paper, a test execution and a test automation effort estimation model are
     discussed.




   12/14/2012                                                                            6
Literature Review…

Methodology                      Strength                          Weakness




Test effort estimation model     Reducing ambiguity.               Here difficult to evaluate team
                                                                   capacity .




          This paper gives an idea how to estimate test effort of test suite.



    12/14/2012                                                                              7
Literature Review…

Paper # 02
Title: Software Test Effort Estimation

Author: D.S.Kushwaha and A.K.Misra, ACM SIGSOFT Software Engineering Notes,
        vol.33,no.3,pp.1-6,2008.

                                            Proposal

                In this paper proposed approach is for software test effort estimation.




   12/14/2012                                                                             8
Literature Review…


Methodology                   Strength                        weakness



CICM(Cognitive Information    CICM is a more robust metric    Not use for large project.
Complexity Measure)           and estimating the software
                              testing effort.




   In this paper CICM can be used as a tool to estimate the test effort when a new component
  is added to existing software.

   12/14/2012                                                                              9
Literature Review…
 Paper#3
 Title: An Experience-Based Approach For Test Execution Effort Estimation

  Author: X.Zhu, B. Zhou, L.Hou, J.Chen and Lu Chen,IEEE The 9th International
         Conference for Young Computer Scientists,pp.1193-1198,2008.

                                        Proposal



              This paper characterize a test suite as a 3-dimensions vector.




 12/14/2012                                                                    10
Literature Review…

Methodology                                Strength                Weakness




                                                                   Not Predict Test Case Number
Test Suite Execution Vector               Low cost                 from early time based in
                                                                   requirements.




       In this paper estimating test execution effort for both new tests and regression tests.


    12/14/2012                                                                              11
Literature Review…
Paper#4
Title: Software Testing Effort: An Assessment Through Fuzzy Criteria Approach

Author: P. R.Srivastava, S.Kumar, A.P. Singh and G.Raghurama ,Journal of
        Uncertain Systems,vol.5,no.3,pp.183-201,2011.


                                            Proposal


            This paper makes an attempt to estimate software testing effort using
            fuzzy logic.




    12/14/2012                                                                      12
Literature Review…


Methodology                      Strength                          Weakness




                                 By this process software          Determining of fuzzy rules
Fuzzy logic                      testing efforts can be            which depends on the
                                 evaluated more accurately         experience of the decision
                                                                   maker.




         In this paper test driven are used which are useful for test effort estimation.


   12/14/2012                                                                              13
Literature Review…
Paper#5
Title: Applying Requirement Based Complexity For The Estimation Of Software
       Development And Testing Effort.

Author: A.Sharma and D.S.Kushwaha ,Computer Science & Information Technology,
        pp.393–415, 2012.


                                        Proposal

  This paper presents a systematic and an integrated approach for the estimation of
  software development and testing effort on the basis of improved requirement based
  complexity (IRBC) of the proposed software.




    12/14/2012                                                                         14
Literature Review…


Methodology                   Strength                        Weakness


                              Reducing rework by              If the result of IRBC is
RBTEE(Requirement based       delivering maximum              incorrect than RBTEE
test effort estimate )        coverage with minimum           estimation also produce
                              number of test cases for        incorrect result.
                              improving the test
                              effectiveness.




    This paper gives an idea How to estimate test effort on the basis of requirement.




  12/14/2012                                                                             15
Critical analysis of test effort estimation
Parameters      Test Effort                      An Experience-     Software          Applying Requirement
   of           Estimation          Software     Based Approach     Testing Effort:   Based
evaluation      Models Based on    Test Effort   for Test           An Assessment     Complexity For The
                Test               Estimation    Execution Effort   Through           Estimation Of
                Specification[1]   [2]           Estimation[3]      Fuzzy Criteria    Software Development
                                                                    Approach[4]       And Testing Effort[5]


LOC             less               more          less               more              more


Requirement     no                 no            no                 yes               yes


Time            high               medium        high               high              low



Cost            high               medium        high               high              less


   12/14/2012                                                                                        16
Module:2 Regression testing


  “Regression testing is an important and necessary activity that can maintain the
    quality of modified software”[8].



   “Regression testing detecting newly introduced faults in modified software, is
    an expensive part of the software maintenance process”[9].




   12/14/2012                                                                   17
Literature Review

Paper#(1)
Title: Code Coverage Based Technique For Prioritizing Test Cases For Regression
       Testing

Author: K.K.Aggrawal, Y.Singh and A.Kaur, ACM SIGSOFT Software Engineering Notes ,
        vol.29 ,no.5,pp.1-4,2004.

                                    proposal


       In this paper discussed, A model that achieves 100% code coverage optimally
      during regression testing.




    12/14/2012                                                                       18
Literature Review…

 Methodology                     Strength                         Weakness




                                 It save the cost and effort by   Use literature for test case
 Code coverage based             removing extra test cases.       prioritization.




 This paper gives an idea of test case prioritization which is technique of regression testing.




  12/14/2012                                                                               19
Literature Review…
Paper#(2)
Title: Historical Value-Based Approach for Cost-cognizant Test Case Prioritization
       To Improve The Effectiveness Of Regression Testing.

Author: H.Park, H.Ryu and J.Baik, IEEE The Second International Conference on Secure
       System Integration And Reliability Improvement ,pp.39-46,2008.


                                           Proposal


          In this paper authors proposed the Historical Value-Based Approach.




    12/14/2012                                                                       20
Literature Review…



 Methodology                         Strength                   weakness
HVBA (Historical value based   Useful and effective   Only give priority for test case
approach)                                             not for fault severity.




    12/14/2012                                                                 21
Literature Review…

Paper#(3)
Title: Automating Regression Test Selection Based On UML Designs

Author: Automating regression test selection based on UML designs , Information and
        Software Technology, pp.16-30, 2009.

                                    Proposal

       In this paper author proposes a formal mapping between design changes and a
      classification of regression test cases.




    12/14/2012                                                                       22
Literature Review…

Methodology                    Strength                    Weakness




UML tool to track the          Higher efficiency in test   UML design information ,test
regression test selection at   selection                   selection may not be precise
architecture level.                                        as it was detailed code
                                                           analysis.




   12/14/2012                                                                    23
Literature Review…
Paper#(4)
Title: A Hybrid Approach For Regression Testing In Interprocedural Program

Author: Y.Singh, A.Kaur and B.Suri, Journal of Information Processing Systems,
        Vol.6, No.1,pp.22-32,2010.
                                 Proposal

  The proposed hybrid approach is based on the selection and prioritization of the test
  cases.




  12/14/2012                                                                              24
Literature Review…




  Methodology             Strength                        Weakness

Hybrid approach      Produce less no of test case   Not build the new test case.




    12/14/2012                                                              25
Literature Review…
Paper#(v)
Title: Spectrum-Based Fault Localization in Regression Testing

Author: S.Tiwari. K. K. Mishra, A.Kumar and A.K. Misra, Eighth International Conference
        on Information Technology: New Generations IEEE,pp.191-195,2011.


                                            proposal

     This paper addresses the fundamental limitations of conventional regression testing
     approach.




   12/14/2012                                                                          26
Literature Review…



     Methodology       Strength   Weakness

Spectrum-Based Fault    Simple    Oracle problem
Localization




    12/14/2012                                     27
Critical Analysis Of Regression Testing
Parameters     Code           Historical       Automatin     A Hybrid        Spectrum-
    of         Coverage       Value-Based      g             Approach for    Based Fault
evaluation     Based          Approach for     regression    Regression      Localization
               Technique      Cost-cognizant   test          Testing in      in Regression
               For            Test Case        selection     Interprocedur   Testing[5]
               Prioritizing   Prioritization   based         al Program[4]
               Test Cases     to Improve        on
               For             the             UML
               Regression     Effectiveness     designs[3]
                Testing [1]   of
                              Regression
                              Testing[2]

Time           Medium         Medium           Low           High            Low

Cost           Medium         Medium           Low           High            Low

complexity     Less           High             Less          High            Less
  12/14/2012                                                                          28
Conclusion

• Methods discussed are based on test effort estimation and regression testing.

• We will work on the approach for regression test selection which is a
  technique of regression testing.




    12/14/2012                                                                29
References
[1] O.Mizuno,E.Shigematsu ,Y.Takagi, and T.Kikuno “On Estimating Testing Effort Needed to Assure Field
     Quality in Software Development” IEEE Proceedings of the 13 the International Symposium on
     Software Reliability Engineering , vol. 54, no. 2, pp.1-8, 2002.
[2] E.Aranha and P.Borba “Test Effort Estimation Models Based on Test Specifications” , IEEETesting:
     Academic and Industrial Conference - Practice And Research Techniques ,pp.67-71,2007.
[3] D.S.Kushwaha and A.K.Misra “ Software Test Effort Estimation ”, ACM SIGSOFT Software Engineering
     Notes, vol.33 ,no.3,pp.1-6,2008.
[4] X.Zhu, B. Zhou, L.Hou, J.Chen and Lu Chen “An Experience-Based Approach for Test Execution Effort
     Estimation”, IEEE The 9th International Conference for Young Computer Scientists ,pp.1193-1198,2008.
[5] P. R.Srivastava, S.Kumar, A.P. Singh and G.Raghurama “Software Testing Effort: An Assessment Through
     Fuzzy Criteria Approach”, Journal of Uncertain Systems ,vol.5, no.3, pp.183-201,2011.
[6].A. Sharma and D.S.Kushwaha “Applying Requirement Based Complexity For The Estimation Of Software
     Development And Testing Effort", Computer Science & Information Technology ,pp. 393–415,2012.
[7]Y.C.Huangc ,K.L.Pengaand and C.Y. Huanga,b “A history-based cost-cognizant test case prioritization
     technique in regression Testing” ,The Journal of Systems and Software 85 , vol.05,no.02,pp.626–
     637,2012.
[8] Xiao Qu “Configuration Aware Prioritization Techniques in Regression Testing ”,Indian Computer
     Education Society,vol. 42,no.6,pp.1-4,2009.


     12/14/2012                                                                                    30
References

[9] K.K.Aggrawal, Y.Singh and A.Kaur “Code Coverage Based Technique For Prioritizing Test Cases For
      Regression Testing ” ,ACM SIGSOFT Software Engineering Notes , vol.29 ,no. 5,pp.1-4,2004.
[11] H.Park, H.Ryu and J.Baik“Historical Value-Based Approach for Cost-cognizant Test Case Prioritization
    to Improve the Effectiveness of Regression Testing”,IEEEThe Second International Conference on Secure
     System Integration and Reliability Improvement IEEE,pp.39-46,2008.
[10] L.C. Briand,Y. Labiche,and S. Hea “Automating regression test selection based on UML designs ”
    Information and Software Technology,pp.16-30,2009
[12] Y.Singh, A.Kaur and B.Suri “A Hybrid Approach for Regression Testing in Interprocedural” Program
    Journal of Information Processing Systems ,vol.6, no.1,pp.22-32,2010
[13] S.Tiwari. K. K. Mishra, A.Kumar and A.K. Misra “ Spectrum-Based Fault Localization in Regression
    Test”,IEEE Eighth International Conference on Information Technology: New Generations ,pp.191-
    195,2011.




     12/14/2012                                                                                   31

software

  • 1.
    Second Progress Presentation On Test Effort Estimation In Regression Testing Presented By: Supervisor: Abhilasha, Mr. Ashish Sharma, M. Tech, Department of CEA, Enroll :118150001. GLA university.
  • 2.
    Outline  Problem Statement  Literature review  Critical Analysis  Conclusion  References 12/14/2012 2
  • 3.
    Problem Statement: “Test Effort Estimation In Regression Testing” 12/14/2012 3
  • 4.
    Literature review The research work is formulated in two modules : -Test Effort Estimation -Regression Testing 12/14/2012 4
  • 5.
    Module I :TestEffort Estimation • Test effort estimation is the estimation of the testing size, testing cost and testing schedule for a specified software test project by defined methods, tools and techniques[1]. • The test effort is defined as the sum of the effort spent on test activity and the effort spent on debug activity [2]. 12/14/2012 5
  • 6.
    Literature Review Paper#1 Title: Test Effort Estimation Models Based On Test Specifications Author: Eduardo Aranha,and Paulo Borba, IEEE Testing: Academic and Industrial Conference - Practice And Research Technique,pp.67-71,2007. Proposal In this paper, a test execution and a test automation effort estimation model are discussed. 12/14/2012 6
  • 7.
    Literature Review… Methodology Strength Weakness Test effort estimation model Reducing ambiguity. Here difficult to evaluate team capacity . This paper gives an idea how to estimate test effort of test suite. 12/14/2012 7
  • 8.
    Literature Review… Paper #02 Title: Software Test Effort Estimation Author: D.S.Kushwaha and A.K.Misra, ACM SIGSOFT Software Engineering Notes, vol.33,no.3,pp.1-6,2008. Proposal In this paper proposed approach is for software test effort estimation. 12/14/2012 8
  • 9.
    Literature Review… Methodology Strength weakness CICM(Cognitive Information CICM is a more robust metric Not use for large project. Complexity Measure) and estimating the software testing effort. In this paper CICM can be used as a tool to estimate the test effort when a new component is added to existing software. 12/14/2012 9
  • 10.
    Literature Review… Paper#3 Title: An Experience-Based Approach For Test Execution Effort Estimation Author: X.Zhu, B. Zhou, L.Hou, J.Chen and Lu Chen,IEEE The 9th International Conference for Young Computer Scientists,pp.1193-1198,2008. Proposal This paper characterize a test suite as a 3-dimensions vector. 12/14/2012 10
  • 11.
    Literature Review… Methodology Strength Weakness Not Predict Test Case Number Test Suite Execution Vector Low cost from early time based in requirements. In this paper estimating test execution effort for both new tests and regression tests. 12/14/2012 11
  • 12.
    Literature Review… Paper#4 Title: SoftwareTesting Effort: An Assessment Through Fuzzy Criteria Approach Author: P. R.Srivastava, S.Kumar, A.P. Singh and G.Raghurama ,Journal of Uncertain Systems,vol.5,no.3,pp.183-201,2011. Proposal This paper makes an attempt to estimate software testing effort using fuzzy logic. 12/14/2012 12
  • 13.
    Literature Review… Methodology Strength Weakness By this process software Determining of fuzzy rules Fuzzy logic testing efforts can be which depends on the evaluated more accurately experience of the decision maker. In this paper test driven are used which are useful for test effort estimation. 12/14/2012 13
  • 14.
    Literature Review… Paper#5 Title: ApplyingRequirement Based Complexity For The Estimation Of Software Development And Testing Effort. Author: A.Sharma and D.S.Kushwaha ,Computer Science & Information Technology, pp.393–415, 2012. Proposal This paper presents a systematic and an integrated approach for the estimation of software development and testing effort on the basis of improved requirement based complexity (IRBC) of the proposed software. 12/14/2012 14
  • 15.
    Literature Review… Methodology Strength Weakness Reducing rework by If the result of IRBC is RBTEE(Requirement based delivering maximum incorrect than RBTEE test effort estimate ) coverage with minimum estimation also produce number of test cases for incorrect result. improving the test effectiveness. This paper gives an idea How to estimate test effort on the basis of requirement. 12/14/2012 15
  • 16.
    Critical analysis oftest effort estimation Parameters Test Effort An Experience- Software Applying Requirement of Estimation Software Based Approach Testing Effort: Based evaluation Models Based on Test Effort for Test An Assessment Complexity For The Test Estimation Execution Effort Through Estimation Of Specification[1] [2] Estimation[3] Fuzzy Criteria Software Development Approach[4] And Testing Effort[5] LOC less more less more more Requirement no no no yes yes Time high medium high high low Cost high medium high high less 12/14/2012 16
  • 17.
    Module:2 Regression testing “Regression testing is an important and necessary activity that can maintain the quality of modified software”[8]. “Regression testing detecting newly introduced faults in modified software, is an expensive part of the software maintenance process”[9]. 12/14/2012 17
  • 18.
    Literature Review Paper#(1) Title: CodeCoverage Based Technique For Prioritizing Test Cases For Regression Testing Author: K.K.Aggrawal, Y.Singh and A.Kaur, ACM SIGSOFT Software Engineering Notes , vol.29 ,no.5,pp.1-4,2004. proposal In this paper discussed, A model that achieves 100% code coverage optimally during regression testing. 12/14/2012 18
  • 19.
    Literature Review… Methodology Strength Weakness It save the cost and effort by Use literature for test case Code coverage based removing extra test cases. prioritization. This paper gives an idea of test case prioritization which is technique of regression testing. 12/14/2012 19
  • 20.
    Literature Review… Paper#(2) Title: HistoricalValue-Based Approach for Cost-cognizant Test Case Prioritization To Improve The Effectiveness Of Regression Testing. Author: H.Park, H.Ryu and J.Baik, IEEE The Second International Conference on Secure System Integration And Reliability Improvement ,pp.39-46,2008. Proposal In this paper authors proposed the Historical Value-Based Approach. 12/14/2012 20
  • 21.
    Literature Review… Methodology Strength weakness HVBA (Historical value based Useful and effective Only give priority for test case approach) not for fault severity. 12/14/2012 21
  • 22.
    Literature Review… Paper#(3) Title: AutomatingRegression Test Selection Based On UML Designs Author: Automating regression test selection based on UML designs , Information and Software Technology, pp.16-30, 2009. Proposal In this paper author proposes a formal mapping between design changes and a classification of regression test cases. 12/14/2012 22
  • 23.
    Literature Review… Methodology Strength Weakness UML tool to track the Higher efficiency in test UML design information ,test regression test selection at selection selection may not be precise architecture level. as it was detailed code analysis. 12/14/2012 23
  • 24.
    Literature Review… Paper#(4) Title: AHybrid Approach For Regression Testing In Interprocedural Program Author: Y.Singh, A.Kaur and B.Suri, Journal of Information Processing Systems, Vol.6, No.1,pp.22-32,2010. Proposal The proposed hybrid approach is based on the selection and prioritization of the test cases. 12/14/2012 24
  • 25.
    Literature Review… Methodology Strength Weakness Hybrid approach Produce less no of test case Not build the new test case. 12/14/2012 25
  • 26.
    Literature Review… Paper#(v) Title: Spectrum-BasedFault Localization in Regression Testing Author: S.Tiwari. K. K. Mishra, A.Kumar and A.K. Misra, Eighth International Conference on Information Technology: New Generations IEEE,pp.191-195,2011. proposal This paper addresses the fundamental limitations of conventional regression testing approach. 12/14/2012 26
  • 27.
    Literature Review… Methodology Strength Weakness Spectrum-Based Fault Simple Oracle problem Localization 12/14/2012 27
  • 28.
    Critical Analysis OfRegression Testing Parameters Code Historical Automatin A Hybrid Spectrum- of Coverage Value-Based g Approach for Based Fault evaluation Based Approach for regression Regression Localization Technique Cost-cognizant test Testing in in Regression For Test Case selection Interprocedur Testing[5] Prioritizing Prioritization based al Program[4] Test Cases to Improve on For the UML Regression Effectiveness designs[3] Testing [1] of Regression Testing[2] Time Medium Medium Low High Low Cost Medium Medium Low High Low complexity Less High Less High Less 12/14/2012 28
  • 29.
    Conclusion • Methods discussedare based on test effort estimation and regression testing. • We will work on the approach for regression test selection which is a technique of regression testing. 12/14/2012 29
  • 30.
    References [1] O.Mizuno,E.Shigematsu ,Y.Takagi,and T.Kikuno “On Estimating Testing Effort Needed to Assure Field Quality in Software Development” IEEE Proceedings of the 13 the International Symposium on Software Reliability Engineering , vol. 54, no. 2, pp.1-8, 2002. [2] E.Aranha and P.Borba “Test Effort Estimation Models Based on Test Specifications” , IEEETesting: Academic and Industrial Conference - Practice And Research Techniques ,pp.67-71,2007. [3] D.S.Kushwaha and A.K.Misra “ Software Test Effort Estimation ”, ACM SIGSOFT Software Engineering Notes, vol.33 ,no.3,pp.1-6,2008. [4] X.Zhu, B. Zhou, L.Hou, J.Chen and Lu Chen “An Experience-Based Approach for Test Execution Effort Estimation”, IEEE The 9th International Conference for Young Computer Scientists ,pp.1193-1198,2008. [5] P. R.Srivastava, S.Kumar, A.P. Singh and G.Raghurama “Software Testing Effort: An Assessment Through Fuzzy Criteria Approach”, Journal of Uncertain Systems ,vol.5, no.3, pp.183-201,2011. [6].A. Sharma and D.S.Kushwaha “Applying Requirement Based Complexity For The Estimation Of Software Development And Testing Effort", Computer Science & Information Technology ,pp. 393–415,2012. [7]Y.C.Huangc ,K.L.Pengaand and C.Y. Huanga,b “A history-based cost-cognizant test case prioritization technique in regression Testing” ,The Journal of Systems and Software 85 , vol.05,no.02,pp.626– 637,2012. [8] Xiao Qu “Configuration Aware Prioritization Techniques in Regression Testing ”,Indian Computer Education Society,vol. 42,no.6,pp.1-4,2009. 12/14/2012 30
  • 31.
    References [9] K.K.Aggrawal, Y.Singhand A.Kaur “Code Coverage Based Technique For Prioritizing Test Cases For Regression Testing ” ,ACM SIGSOFT Software Engineering Notes , vol.29 ,no. 5,pp.1-4,2004. [11] H.Park, H.Ryu and J.Baik“Historical Value-Based Approach for Cost-cognizant Test Case Prioritization to Improve the Effectiveness of Regression Testing”,IEEEThe Second International Conference on Secure System Integration and Reliability Improvement IEEE,pp.39-46,2008. [10] L.C. Briand,Y. Labiche,and S. Hea “Automating regression test selection based on UML designs ” Information and Software Technology,pp.16-30,2009 [12] Y.Singh, A.Kaur and B.Suri “A Hybrid Approach for Regression Testing in Interprocedural” Program Journal of Information Processing Systems ,vol.6, no.1,pp.22-32,2010 [13] S.Tiwari. K. K. Mishra, A.Kumar and A.K. Misra “ Spectrum-Based Fault Localization in Regression Test”,IEEE Eighth International Conference on Information Technology: New Generations ,pp.191- 195,2011. 12/14/2012 31