1. • All process should have metrics.
• Metrics in testing domain – track test
progress ,evaluate quality , test
effectiveness & when to stop testing .

2.  Meaningful metrics begin with questions –
 Which measure to collect?
 Purpose ?
 Who collects them ?
 Forms & Tools used ?
 Who analyses the data ?
 Who has the access to report ?

3.  Begin with a test plan .
 Work done is compared with the work
planned
 To support monitoring , Test plans should
contain Testing milestones .

4.  Milestones are tangible events that are
expected to occur at a certain time in the
project’s lifetime.
 Gives the status of the project .

5. • Test Milestones can be used to monitor the
progress of the testing efforts associated
with a software project .
• Guideposts or Goals that needs to be met .
• Test Manager uses current testing effort data
to find how close the milestone achievement
is

6. • Milestone usually appear in the Scheduling
Component of the test plan.
• Different levels of testing have different
milestones .

7.  Completion of the master test plan ?
 Completion of Branch Coverage ? ( Unit test )
 Execution of all planned system tests ?
 Completion of Test summary Report ?

8.  Every milestone has to be scheduled for
completion during a certain period of time
 A group of test members are assigned for
achieving the milestone on time and allotted
budget .

9.  Test planners need to be sure that milestone
are plausible & meaningful .
 Milestones should be descripted in simple
terms and ambiguities should be avoided.
 Monitoring Data are collected that relates to
the current status of the testing cycle .

10. • Graphs Can be used to show the trends or
the phases of the monitoring process over a
limited period of time .
• The graphs can be in form of bar charts and
can be represented for days , weeks or
months based on the scalability of the test to
be done .

11.  Project & test leaders present up-to-date
measurements , graphs and plots showing the
testing efforts.
 Testing milestones achieved and those which
are not .
 Test logs , test incident reports & other test-
related documents
 Managers question the team .

12.  Testing Status
 Tester Productivity
 Testing costs
 Errors , Faults & Failures .

13.  Identifying the current state of the testing
process .
 Manager needs to determine if the testing
tasks are being completed on time and
within budget .

14. • Which tasks are on time ?
• Which are behind Schedule , and by how
much ?
• Have the scheduled milestones for this date
been met ?

15.  Test Mangers measures how much coverage
has been actually achieved and how much is
left .
 Compare it with the planned coverage.

16. • White box testing – Degree of statement ,
branch , data flow , basis path ( planned ,
actual )
• Actual / planned degree of coverage

17.  Black Box testing – Number of requirements
to be tested , equivalence classes
identified , equivalence classes actually
covered
 No of features actually covered/ total no of
features to be covered .

18.  To monitor the progress of test case
development , and can be applied to all
levels of testing .
 Estimated Test cases- No of planned test
cases
 Test cases complete – No of available test
cases .
 Change in requirements – Unplanned test
cases .

19. • Test manager will want to determine if the
execution process is going according to plan .
• Appropriate measures
 No of available test cases executed
 No of available test cases exe & passed
 No of unplanned test cases
 No of unplanned test cases exe and passed .

20.  For new release where there can be
regression testing
 No of planned regression tests exe
 No of planned regression test exe and
planned

21.  Imp that Test managers To monitor progress
of the development of the test harness code
needed for unit and integration test – Timely
manner
 Size – applied by manager to help estimate
the amt of effort needed to develop a s/w
pgm .

22. • Size can be measured in different ways
 Lines of code
 Function points
 Feature points
• Commonly used is Loc

23.  Monitor the test harness development effort
over time – Available LOC / planned LOC .

24.  Productivity of the staff and how it changes
the Project progresses.
 Measuring productivity in s/w domain is
difficult task .
 Since developers are involved in many
activities , many are complex

25.  The existing method of measuring
productivity of developers and testers are
not credible .
 Productivity of tester can be measured using
a combination of
 No of test cases developed /unit time
 No of test cases exe
 No of Loc Test Harness developed /unit time
 No of defects detected in unit time.

26.  Good Technique – budget and resource
monitoring is called earned value tracking .
 To calculate Planned earned value
 Total estimated time of budget for overall
testing effort
 Estimated time or budget for each testing
task

27. • Earned value can be calculated separately
for each level of testing
 This will help in monitoring the budget and
resource usage for individual testing phase .
Actual time for each testing task .
Earned value for testing task to date .

28.  Useful for
 Evaluating product quality
 Evaluating testing effectiveness
 Making stop-test decisions .

29.  Defect casual analysis
 Defect prevention
 Test process improvement
 Development process improvement .

30. • Total number of incident reports.
• Number of incident reports resolved /
unresolved.
• Number of defects found of each given type.
• Number of failures .

31. • Allows managers to determine if test
resources have been used wisely and
productivity to remove defects and evaluate
product quality .
• Evaluation is made in several ways ,
• Before release – no of defects found during
testing / number of defects estimated .
• After release – no of defects found / 1000
lines of new and changed code

32. • Defect removal ratio
Pre-ship fault density / post- ship fault
density .

33. • Measurement suggested by Graham
No of defects detected in a given test phase/
total number of defects found in testing.
Detect removal leverage = no of defects
found integration test / no of defects found
in unit test .