The document presents a method for estimating defects remaining after a series of software inspections using minimal additional process steps and collected data. It involves inspectors privately recording defects found before each inspection and identifying defects they previously found during the meeting. A capture-recapture metric is calculated and used with tables or charts along with total defects found to predict remaining defects and establish confidence limits, as validated through simulations. The method aims to provide useful estimates with minimal effort for projects lacking metrics experience.

1. Minimum Process Requirements
To Predict Defects Missed
In A Series Of Software Inspections
James K. Orr
Independent Consultant
jkorr@gatech.edu
1Copyright 2015 By James K. Orr

2. Objective
• Provide as accurate as possible estimate of
defects remaining after a series of software
inspections (for example, all inspections on an
upcoming release) based on the minimum
amount of additional process activity and
information collected from the inspection
meeting.
2Copyright 2015 By James K. Orr

3. Contents
Section Page
Application Of Process Example 5
Prediction Charts For 100 and 200 Defects Found 10
Tables And Plots To Compute Number Of Defects
Found And Confidence Limits
13
Simulation Basis Inspection Team Effectiveness As A
Function Of Capture – Recapture Metric
19
Simulation Basis For Number Of Defects Found And
Confidence Limits
23
Conclusions 27
3Copyright 2015 By James K. Orr

4. Application Of Process
Example
4Copyright 2015 By James K. Orr

5. Additional Inspection Process Steps
• Additional Step At Start Of Inspection
– Moderator asks each inspector how many defects were
found prior to meeting and temporarily records
information for use at end of meeting.
• Additional Step At End Of Inspection (Moderator has
previously determined how many valid defects were
found).
– For each valid defect, the Moderator asks inspectors to
indicate if they found the defect prior to the inspection
meeting. Moderator insures data from each inspector is
consistent with information from the start of meeting.
• Moderator provides a coordinator with the information
collected in the format on the following page.
5Copyright 2015 By James K. Orr

6. Additional Information Collected
Inspection
Identification
Information
Defects Found Pre-
Inspection By Only
One Inspector
Defects Found Pre-
Inspection By Two
Or More Inspectors
Additional Defects
Found During The
Meeting
Inspection 105 3 1 0
6Copyright 2015 By James K. Orr

7. Additional Metrics Coordinator Process
• A metric is computed for use in a chart or
table lookup.
– Capture-Recapture Metric = (Sum Of Errors Found
By Two Or More Inspectors) / (Sum Of Errors
Found By Only One Inspector)
• Lookup Predicted Defects Missed By Multiple
Inspections Based On Total Number Of Errors
Found and Capture-Recapture Metric
7Copyright 2015 By James K. Orr

8. Metrics Coordinator Calculation
Inspection
Identification
Information
Defects Found Pre-
Inspection By Only
One Inspector
Defects Found Pre-
Inspection By Two
Or More Inspectors
Additional Defects
Found During The
Meeting
Inspection 105 3 1 0
Inspection 106 3 1 0
Inspection 105 2 2 1
Inspection 107 4 0 0
Inspection 108 2 1 0
Inspection 109 3 3 0
Sum 17 8 1
Calculate Capture-
Recapture Metric
Value Of Capture-
Recapture Metric
Predicted Defects Missed
By Multiple Inspections
= 8 / 17 47.06 %
For 25 Errors Found
18
See Next Page
8Copyright 2015 By James K. Orr

9. Prediction Of Defects Remaining
Total Of 25 Defects Found
9Copyright 2015 By James K. Orr

10. Prediction Charts For 100
and 200 Defects Found
10Copyright 2015 By James K. Orr

11. Prediction Of Defects Remaining
Total Of 100 Defects Found
Copyright 2015 By James K. Orr 11

12. Prediction Of Defects Remaining
Total Of 200 Defects Found
Copyright 2015 By James K. Orr 12

13. Tables And Plots To Compute
Number Of Defects Found And
Confidence Limits
13Copyright 2015 By James K. Orr

14. Table Of Factor Based On
Capture – Recapture Metric
Copyright 2015 By James K. Orr 14

15. Plot Of Factor Based On
Capture – Recapture Metric
Copyright 2015 By James K. Orr 15

16. Table Of Factor Based On
Number Of Defects Found
Copyright 2015 By James K. Orr 16

17. Plot Of Factor Based On
Number Of Defects Found
Copyright 2015 By James K. Orr 17

18. Sample Calculation
Copyright 2015 By James K. Orr 18
Calculate
Capture-
Recapture
Metric
Value Of
Capture-
Recapture
Metric
Uncorrected
Defects
Missed
Divided By
Defect Found
In Inspection
Number Of
Defects
Found In
Inspection
Adjusted
Expected
Value
Factor
Number Of
Defects Missed
In Inspection
= 8 / 17 47.06 % 0.656
Calculated,
Not From
Table
25 109% = integer (25 *
0.656 *1.09)
=integer( 17.9)
= 18
• Note: Detain analysis done for 25, 100, and 200 defects
found in inspections.
• Values in Table are interpolated for intermediate values.

19. Simulation Basis
Inspection Team Effectiveness
As A Function Of
Capture – Recapture Metric
19Copyright 2015 By James K. Orr

20. Team Inspection Effectiveness
As A Function Of Capture – Recapture Metric
Copyright 2015 By James K. Orr 20

21. Simulation To Develop
One Data Point On Prior Chart
Copyright 2015 By James K. Orr 21

22. Convert To Missed Defects
Defects
Missed
Copyright 2015 By James K. Orr 22
=
Defects
Found *
1 - 1( )_______________
Verification Effectiveness

23. Simulation Basis
For Number Of Defects Found
And Confidence Limits
23Copyright 2015 By James K. Orr

24. Prediction Error Analysis
25 Defects Found
Copyright 2015 By James K. Orr 24
Legend:
Insp 4 Eff 20 %
Means Simulation
With 4 Inspectors
Each with Individual
Defect Detection of
20 % Of Errors In
Inspected Product

25. Prediction Error Analysis
100 Defects Found
Copyright 2015 By James K. Orr 25
Legend:
Insp 6 Eff 15 %
Means Simulation
With 6 Inspectors
Each with Individual
Defect Detection of
15 % Of Errors In
Inspected Product

26. Prediction Error Analysis
200 Defects Found
Copyright 2015 By James K. Orr 26
Legend:
Insp 4 Eff 20% to 25 %
Means Simulation
With 4 Inspectors
Each with Individual
Defect Detection of
Random From 20 % to
25 % Of Errors In
Inspected Product

27. Conclusions
• A detail, simple method has been presented which will
allow a project to perform prediction of defects missed in
a series of software inspections with minimum effort.
• For projects with minimal software metric experience,
useful data can be produced with very minor effort.
• For projects with extensive software metric experience,
the project can begin to evaluate usefulness of capture-
recapture metrics with minimum resistance due to
collection of project data without individual inspector
information (used by moderator in the inspection team
meeting only).
Copyright 2015 By James K. Orr 27