I have been thinking since a while to derive a math around defect predictability. With lot of thought process, I have come up with a Simple Approach backed by a complex math in order to predict defect rate for a release/feature, in advance to implementation/test cycles.
2. Antecedent
Can we predict quality based on historical data?
Can someone answer this $M question – How many defects per feature,
before even test cycles commence?
Will defect prediction help in better estimation and release planning?
Will discovering likelihood of defect density help in functional coverage?
Will anticipation of defect discovery rate trigger proactive thoughts to
avoid the same?
3. Parameters to consider
Complexity
Interfaces
Frequency of usage and User importance
Non-functional aspects (if any)
Engineering capabilities & Experience (domain and
tools)
Environmental/Platform considerations
(Optional/Subjective) impact on legacy system
4. Complexity
1. (R)equirements (scale: 1-5) – How tough or unambiguous
or clear are they, to understand?
2. (D)esign (1-4) – How complex is that?
3. (I)mplementation (1-3) – What level of expertize is
needed?
Note: Variation in scale is intentional and is based on phase of feature delivery. I.e, earlier the understanding of complexity better
will be the forecast calculation.
Overall Complexity Score [C]: (R^D)^I
5. Interfaces
1. # of (I)ntegration points with sub-systems (DB/File management,
other co-existing features: 1-5)
2. (D)epth of crossing points (heavy data transfers/ops?: 1-4)
3. Critical (O)perations on edges (Read/Write: 1-3)
Note:Variation in scale is by design and is based on potential failure rate for each of the sub-parameters
Overall Score [IF]: (I^O)*D
6. Freq. of Usage & Importance
1. How (F)requently the feature will be accessed by an end
user (1-5)?
2. Will this feature (I)ndependently consumed or as part of
workflows (1-4)?
Note:Variation in scale is intentional and is based on potential failure rate for each of the sub-parameters
Overall Score [FQ]: F*I
7. Non-functional aspects
1. Any of the below non-functional parameters to be
considered (below list is subjective to nature of the
product and specific priorities)?
(S)ecurity (1-5)
(U)sability (1-2)
S(C)ale (1-5)
(P)erformance & Efficiency (1-5)
Note:Variation in scale is intentional and is based on the impact that each non-functional requirement can have to cause possible
defects in the system
Overall Score [NF]: S+U+C+P
8. Engg. Capabilities/expertize
1. Do all engineers working on product under consideration have
right (C)apabilities, in terms of tools/programming language (1-3)?
2. Do we have right set of engineers who are (D)omain experts (1-
3)?
3. Does the team has similar (E)xperience w.r.t functionality that’s
being developed (1-3)?
4. Are we (R)eusing the legacy code (1-3)?
Note: Higher the overall score for this parameter, lower will be the defects in the system.
Overall Score [CAP]: (C+D+E+R)*# of engineers working on the feature
9. Environmental/platform
1. How many (p)latforms to support (1-5)?
(1: ONE platform; 2: <5 platforms; 3: < 10; 4: <20; 5: >20)
2. How many (c)onfigurations to consider (1-5: as per standards for #1)?
3. How many flavors of (S)W to keep in mind (1-5: DB, Browsers, App/Web
servers)?
4. Is this (T)ightly coupled with co-existing sub-systems (1-3)?
Note: Probability of higher rate of possible defects is high, if the overall score is significant
Overall Score [ENV]: (p*c)+S+T
10. (Optional/Subj) impact on legacy s/m
1. (I)ntensity of impact on legacy code (1-3: regression)
Note: Consider collateral damage that the feature under consideration can cause to existing sub-systems/code
Score [LEG]: I
11. The math
# of possible defects (per feature: FEAT) = (((C*IF)+FQ)*NF+ENV+LEG)/CAP
# forecast of defects for the entire release =
FEAT(1) + FEAT(2)+ …FEAT(N)
12. Conclusion
1. The final math is based on various factors such as but not limited to –
Avg. of historical data across various domains
Possible intensity of each parameter on overall score
Weightage for each of the attributes under consideration
Importance of every pointer across the industry
