The document discusses the importance of metrics and evaluations in software testing, focusing on how they derive insights from raw data to aid decision-making. It outlines steps for metrics analysis, types of metrics, and various calculations involved in tracking project performance, product quality, and defect management. Overall, it emphasizes the role of metrics in ensuring effective testing and timely product releases while highlighting essential metrics for measuring productivity and defects.

1.  ASSIGNMENT ON METRICS AND EVALUATIONSASSIGNMENT ON METRICS AND EVALUATIONS
 SUBMITTED BY:SUBMITTED BY:
 Pavithra.S.Pavithra.S.
 Preethi SubramanianPreethi Subramanian
 Punithavathi.P.Punithavathi.P.
 VidhyaLakshmi.R.VidhyaLakshmi.R.
 YaminiPriya.K.YaminiPriya.K.
SOFTWARE TESTING

2. WHY METRICS AND MEASUREMENT?WHY METRICS AND MEASUREMENT?
 Metrics derive Info from raw data with a view to help in Decision makingMetrics derive Info from raw data with a view to help in Decision making
in areas likein areas like
 Relationship between data pointsRelationship between data points
 Any cause and effect correlation between observed data pointsAny cause and effect correlation between observed data points
 Any pointers to how data can be used in FutureAny pointers to how data can be used in Future
 Project Performance can be tracked by:Project Performance can be tracked by:
 Measuring the Right ParametersMeasuring the Right Parameters
 Performing right analysis on data measuredPerforming right analysis on data measured
 Presenting the results of analysis in an appropriate mannerPresenting the results of analysis in an appropriate manner

3. STEPS IN METRICSSTEPS IN METRICS
 Identify What to measureIdentify What to measure
 Transform Measurements to metricsTransform Measurements to metrics
 Decide operational requirementsDecide operational requirements
 Perform metrics analysisPerform metrics analysis
 Take actions and follow upTake actions and follow up
 Refine measurements and metricsRefine measurements and metrics

4. WHY METRICS IN TESTINGWHY METRICS IN TESTING
 It is essential to measure the progress of testing and product qualityIt is essential to measure the progress of testing and product quality
 Test case execution productivity:Test case execution productivity:
Days needed to complete testing=total test cases yet to be executedDays needed to complete testing=total test cases yet to be executed
 Estimate of total days:Estimate of total days:
Days needed for release=Max(Days needed for testing &fixing defectsDays needed for release=Max(Days needed for testing &fixing defects
 Release date:Release date:
Total days needed for fixing defects= (Outstanding defects yet to be fixedTotal days needed for fixing defects= (Outstanding defects yet to be fixed
++
Defects that can be found in future test)Defects that can be found in future test)
//
Defect fixing capabilityDefect fixing capability

5. Metrics test help in identifyingMetrics test help in identifying
 When making the releaseWhen making the release
 What is the release:What is the release:
Based on defect density across modules & analysis scope ofBased on defect density across modules & analysis scope of
product can be decided to release the product in timeproduct can be decided to release the product in time
 Whether product is being released with known quality:Whether product is being released with known quality:
Deciding whether product is released with known quality orDeciding whether product is released with known quality or
will function on predictable way in field.will function on predictable way in field.

6. TYPES OF METRICSTYPES OF METRICS
 Project metrics:Project metrics:
Indicates how project is planned and executedIndicates how project is planned and executed
 Progress metrics:Progress metrics:
Tracks how different activities are progressingTracks how different activities are progressing
 Productivity metrics:Productivity metrics:
Takes into account various productivity numbers used forTakes into account various productivity numbers used for
planning and testing activitiesplanning and testing activities

7. PROJECT METRICSPROJECT METRICS
 Projects start with information gatheringProjects start with information gathering
 After info collection Planning And tracking Is doneAfter info collection Planning And tracking Is done
 In planning scope is estimated that is mapped as size estimate whichIn planning scope is estimated that is mapped as size estimate which
in turn is mapped as Effort estimatein turn is mapped as Effort estimate
This isThis is Baselined effort.Baselined effort.
 Two factors to be tracked in each phaseTwo factors to be tracked in each phase
 EffortEffort
 ScheduleSchedule

8.  If something goes wrong with the scope or project progressesIf something goes wrong with the scope or project progresses
the process is re-evaluated. This is calledthe process is re-evaluated. This is called Revised effortRevised effort
 The basic measurements used here are:The basic measurements used here are:
 The different activities,the initial baseline and schedule used forThe different activities,the initial baseline and schedule used for
each process.each process.
 The actual effort and time taken for various activities.The actual effort and time taken for various activities.
 The Revused estimate for effort and schedule .The Revused estimate for effort and schedule .

9.  Effort Variance:Effort Variance:
 Plotting baselined ,revised and actual efforts ,phase-wisePlotting baselined ,revised and actual efforts ,phase-wise
 Substantial difference between Baselined and revise effort point toSubstantial difference between Baselined and revise effort point to
incorrect initial estimation.incorrect initial estimation.
Variance%=[(Actual effort – Revised estimate)/Revised estimate]*100Variance%=[(Actual effort – Revised estimate)/Revised estimate]*100
 Schedule Variance:Schedule Variance:
 Deviation of actual schedule from Estimated scheduleDeviation of actual schedule from Estimated schedule
 Calculated at the end of every milestoneCalculated at the end of every milestone
 Several phases could be active at the same time based on theSeveral phases could be active at the same time based on the
SDLC usedSDLC used

10.  Effort Distribution across phasesEffort Distribution across phases
 Variance calculations help in finding out whether commitments are metVariance calculations help in finding out whether commitments are met
 The distribution effort across each phases can be estimated at time ofThe distribution effort across each phases can be estimated at time of
planningplanning

11. PROGRESS METRICSPROGRESS METRICS
 Projects need to be tracked from two angles:Projects need to be tracked from two angles:
 How well project is doing with respect to effort and scheduleHow well project is doing with respect to effort and schedule
 How well product is meeting with the quality requirements forHow well product is meeting with the quality requirements for
releaserelease
 Defects metrics are classified as:Defects metrics are classified as:
 Test defect metricsTest defect metrics
 Development defect metricsDevelopment defect metrics
 Number of defects found in product are main indicators of qualityNumber of defects found in product are main indicators of quality

12.  TEST DEFINE METRICS:TEST DEFINE METRICS:
 Defect priorityDefect priority  Provides management perspective for order of defectsProvides management perspective for order of defects
 Defect SeverityDefect Severity Provides a perspective of impact of the defectProvides a perspective of impact of the defect
 Defect find rate:Defect find rate:
 Purpose of testing is to find defects early in the cyclePurpose of testing is to find defects early in the cycle
 After certain period of testing and fixing plot the results in a Bell Curve.After certain period of testing and fixing plot the results in a Bell Curve.
 Defect fix rate:Defect fix rate:
 Purpose of development is to fix defects as soon as they arrivePurpose of development is to fix defects as soon as they arrive
 Last minute fixing results in slip of deadlines , product quality and also putLast minute fixing results in slip of deadlines , product quality and also put
development at pressuredevelopment at pressure

13.  Outstanding defects rate:Outstanding defects rate:
 No of outstanding defects =Total defects found – Total defects FixedNo of outstanding defects =Total defects found – Total defects Fixed
 Outstanding defects in a well executed project is nearly Zero.Outstanding defects in a well executed project is nearly Zero.
 Priority outstanding rate:Priority outstanding rate:
 Sometimes defects coming out of testing may be critical and enormous.Sometimes defects coming out of testing may be critical and enormous.
 Correspond to extreme and critical classification of defectsCorrespond to extreme and critical classification of defects
 Some defects of high priority may require change in design orSome defects of high priority may require change in design or
architecture while some defects may take only little time but enormousarchitecture while some defects may take only little time but enormous
efforteffort
 Defect trend:Defect trend:
 Consolidating fix rate, find rate, priority outstanding and outstanding intoConsolidating fix rate, find rate, priority outstanding and outstanding into
one chartone chart

14.  Defects classification trends:Defects classification trends:
 How many are Extreme defectsHow many are Extreme defects
 How many are CriticalHow many are Critical
 How many are importantHow many are important
 Weighted defects trend:Weighted defects trend:
 Helps in quick analysis of defectsHelps in quick analysis of defects
 Weighted defect =[Extreme*5+Critical*4+Important*3+Minor*2+Cosmetic]Weighted defect =[Extreme*5+Critical*4+Important*3+Minor*2+Cosmetic]
 Defect cause distribution:Defect cause distribution:
 Why defects occur and what are root causes of the defectWhy defects occur and what are root causes of the defect
 What areas must be focused for getting more defects in testing.What areas must be focused for getting more defects in testing.

15. Development defects metricsDevelopment defects metrics
 Component wise defect distribution:Component wise defect distribution:
 Important to map defects into different componentsImportant to map defects into different components
 Knowing the component that produce more defects help in fixing them.Knowing the component that produce more defects help in fixing them.
 Defect density and defect removal rate:Defect density and defect removal rate:
 Defect density correlates source code and defectsDefect density correlates source code and defects
 Defects per KLOC=[Total defects in the product]/[total executable lineDefects per KLOC=[Total defects in the product]/[total executable line
of codes in KLOC]of codes in KLOC]
 Defect removal rate= (Defects found by verification activitiesDefect removal rate= (Defects found by verification activities
++
Defects found in unit testing) /(Defects found in test team)Defects found in unit testing) /(Defects found in test team)
**
100100

16.  Age analysis of outstanding defects:Age analysis of outstanding defects:
 Age is the time defects have been waiting to be fixed.Age is the time defects have been waiting to be fixed.
 The time needed to fix a defect is proportional to ageThe time needed to fix a defect is proportional to age
 Introduced reopened defects trend:Introduced reopened defects trend:
 Fixing a defect sometimes may lead to the malfunctioning of aFixing a defect sometimes may lead to the malfunctioning of a
currently working proces.This is called Introduced Defect.currently working proces.This is called Introduced Defect.

17. PRODUCTIVITY METRICSPRODUCTIVITY METRICS
 Purpose:Purpose:
 Estimating for new releaseEstimating for new release
 Finding out how well a team is progressingFinding out how well a team is progressing
 Estimating the number of defectsEstimating the number of defects
 Estimating release date and qualityEstimating release date and quality
 Estimating cost involved in ReleaseEstimating cost involved in Release

18.  Defects per 100 Hours of Testing:Defects per 100 Hours of Testing:
 (Total defects found for a period / Total hours spent to get defects)*100(Total defects found for a period / Total hours spent to get defects)*100
 Test cases Executed per 100 Hours of testing:Test cases Executed per 100 Hours of testing:
 (Total test cases executed for a period / Total Hours spent in last(Total test cases executed for a period / Total Hours spent in last
execution)*100execution)*100
 Test cases Developed per 100 Hours of testing:Test cases Developed per 100 Hours of testing:
 (Total test cases developed for a period / Total hours spent in test case(Total test cases developed for a period / Total hours spent in test case
Development)*100Development)*100
 Defects per 100 test cases:Defects per 100 test cases:
 (Total defects found for a period / Total test cases executed for the(Total defects found for a period / Total test cases executed for the
same period)*100same period)*100

19.  Defects per 100 failed test cases:Defects per 100 failed test cases:
 (Total defects found for a period / Total test case failed due to those(Total defects found for a period / Total test case failed due to those
defects)*100defects)*100
 Closed defect distribution:Closed defect distribution:
 Making sure the defects are fixed and the customer gets qualityMaking sure the defects are fixed and the customer gets quality
assuranceassurance

20. RELEASE METRICSRELEASE METRICS
 Deciding the release of productsDeciding the release of products
 Varies from Product to productVaries from Product to product
 Provides guidelines in making the DecisionProvides guidelines in making the Decision

21. CourtesyCourtesy
““Software testing “ by William StallingsSoftware testing “ by William Stallings
22ndnd
EditionEdition