This document discusses using change genealogies, which model dependencies between code changes, to predict defects. It finds that models using change genealogy metrics outperform those based on code complexity or dependency networks alone, achieving better precision while maintaining close recall. Key metrics include network efficiency and relationships between changes and dependency types. The study confirms that code entities combining functionalities from multiple older changes are more defect-prone.

1. Predicting Defects Using
Change Genealogies
Kim Herzig*, Sascha Just†, Andreas Rau†, Andreas Zeller†
* Microsoft Research, UK
† Saarland University,
Germany

2. Prediction Models
• Goal: determine the likelihood of bugs in
code entities
 Quality assurance limited by time and money.
 Can be helpful for project outsiders.
• Trained on “ground truth”
 Known instances and their properties.
 Idea: learning from past for future.
• Predicting / estimating defect likelihood of
new, unknown code entities

3. Fine-Tuning Prediction Models
Machine Learner
Training Methods
Metrics (independent variables)
Prediction Target

4. (Social) Network Metrics
 Some participants more active and
central than others.
 Are these participants also more
crucial?

5. Code Network Metrics
[2008] Zimmermann and Nagappan: “Predicting Defects using Network Analysis on Dependency Graphs”
10100
10010
1101011000
1001011001
0001010111
1001011001
10100
10010
1101011000
1001011001
0001010111
1001011001
10100
10010
1101011000
1001011001
0001010111
1001011001
10100
10010
1101011000
1001011001
0001010111
1001011001
 Code entities communicate with
each other.
Call graphs do not
change significantly  Use call graph network to
compute network metrics.
over time!
10100
10010
1101011000
1001011001
0001010111
1001011001
10100
10010
1101011000
1001011001
0001010111
1001011001
10100
10010
1101011000
1001011001
0001010111
1001011001
10100
10010
1101011000
1001011001
0001010111
1001011001
 Assumption: “Central binaries tend to
be defect-prone”.

6. Change Network Metrics
Idea: Use dependencies between code changes
 Code changes depend on each
other.
 Central code changes tend to be
crucial.
Change Genealogies
 Assumption: “Code being crucially
changed tend to be defect prone”.

7. Change Genealogies (in a nutshell)
[2013] Kim Herzig: “Mining and Untangling Change Genealogies” (PhD thesis)
Directed graph structure
Method level dependencies
Multi-dimensional (space & time)

8. Change Genealogy Metrics
 EGO network metrics
 Measures the immediate impact of changes on other changes.
 GLOBAL network metrics
 Express the long-term impact of changes on other changes.
 Considering the type of the change
 Adding method definition, modifying method call
 Considering parent age
 How old are the parent changes a change depends on.
Change genealogy metrics must be aggregated to source file level.

9. Experimental Setup
Comparing change genealogies
against:

Code complexity models
(e.g. McCabe)

Code dependency models
(Zimmermann & Nagappan)

Combined network models
(Change genealogy & code dependency network metrics)

10. Experimental Setup
Study subjects
Multiple machine learners

11. Prediction Precision
NM & CGM
Change genealogy metrics
Code dependency network metrics (Zimmermann & Nagappan)
Code complexity metrics

12. Confirmed: Network metrics
outperform complexity metrics.
Change genealogy models report
less false positives (higher precision).
Change genealogy model slightly
more false negatives (lower recall).
Combining network metrics: good
recall but worse precision.

13. Influential Metrics
Network efficiency among the top 10 most influential metrics.
Relationship between changes and type of dependency top 2 metrics (for all projects).
Higher number of old parents the higher the probability to add bugs.
 Code entities combining multiple older functionalities more defect prone.

14. Summary
Adapting social network metrics
to change dependency graphs.
Comparing prediction models.
 Change genealogies are well suited for defect prediction
(better precision, close recall).
 Code entities combining multiple older functionalities more defect prone.