1. MINING
SOFTW
ARE AR
CHIVES
Changes and Bugs
Mining and Predicting Development Activities
Thomas Zimmermann
Saarbrücken, May 26, 2008

2. Software development
Build

3. Collaboration

4. Collaboration
Comm.
Archive

5. Collaboration
Comm. Version
Archive Archive

6. Collaboration
Comm. Version Bug
Archive Archive Database

7. Collaboration
Comm. Version Bug
Archive Archive Database
Mining Software Archives

8. eROSE: Guiding developers
Customers who
bought this item also
bought...
Purchase
History

9. eROSE: Guiding developers
Customers who Developers who
bought this item also changed this function
bought... also changed...
Purchase Version
History Archive

13. eROSE suggests further locations.

14. THIS THESIS .

15. THIS THESIS .
additions analysis architecture archives aspects bug cached calls
changes collaboration complexities component concerns cross-
cutting cvs data defects design development drawing dynamine
eclipse effort evolves failures fine-grained fix fix-inducing
graphs hatari history locate matching method mining
predicting program programmers report repositories
revision software support system taking transactions
version visualizing

16. THIS THESIS .
additions analysis architecture archives aspects bug cached calls
changes collaboration complexities component concerns cross-
cutting cvs data defects design development drawing dynamine
eclipse effort evolves failures fine-grained fix fix-inducing
graphs hatari history locate matching method mining
predicting program programmers report repositories
revision software support system taking transactions
version visualizing

17. Contributions of the thesis
Fine-grained analysis of version archives. 1
Project-specific usage patterns of methods (FSE 2005)
Identification of cross-cutting changes (ASE 2006)
Mining bug databases to predict defects. 2
Dependencies predict defects (ISSRE 2007, ICSE 2008)
Domino effect: depending on defect-prone binaries increases
the chances of having defects (In submission).

18. Contributions of the thesis
Fine-grained analysis of version archives. 1
Project-specific usage patterns of methods (FSE 2005)
Identification of cross-cutting changes (ASE 2006)
Mining bug databases to predict defects. 2
Dependencies predict defects (ISSRE 2007, ICSE 2008)
Domino effect: depending on defect-prone binaries increases
the chances of having defects (In submission).

19. Fine-grained analysis
public void createPartControl(Composite parent) {
...
// add listener for editor page activation
getSite().getPage().addPartListener(partListener);
}
public void dispose() {
...
getSite().getPage().removePartListener(partListener);
}

20. Fine-grained analysis
public void createPartControl(Composite parent) {
...
// add listener for editor page activation
getSite().getPage().addPartListener(partListener);
}
public void dispose() {
...
getSite().getPage().removePartListener(partListener);
}

21. Fine-grained analysis
public void createPartControl(Composite parent) {
...
// add listener for editor page activation
getSite().getPage().addPartListener(partListener);
}
public void dispose() { co-added
...
getSite().getPage().removePartListener(partListener);
}

22. Fine-grained analysis
public void createPartControl(Composite parent) {
...
// add listener for editor page activation
getSite().getPage().addPartListener(partListener);
}
public void dispose() { co-added
...
getSite().getPage().removePartListener(partListener);
}

23. Fine-grained analysis
public void createPartControl(Composite parent) {
... close
// add listener for editor page activation open
getSite().getPage().addPartListener(partListener); println
}
public void dispose() { co-added
...
getSite().getPage().removePartListener(partListener);
} begin

24. Fine-grained analysis
public void createPartControl(Composite parent) {
... close
// add listener for editor page activation open
getSite().getPage().addPartListener(partListener); println
}
public void dispose() { co-added
...
getSite().getPage().removePartListener(partListener);
} begin
Co-added items = patterns

25. Fine-grained analysis

26. Fine-grained analysis
public static final native void _XFree(int address);
public static final void XFree(int /*long*/ address) {
lock.lock();
try {
_XFree(address);
} finally {
lock.unlock();
}
}

27. Fine-grained analysis
public static final native void _XFree(int address);
public static final void XFree(int /*long*/ address) {
lock.lock();
try {
_XFree(address);
} finally {
lock.unlock();
}
}
D IN
NGE IONS
CHA CAT
128 4 LO

28. Fine-grained analysis
public static final native void _XFree(int address);
public static final void XFree(int /*long*/ address) {
lock.lock();
try {
_XFree(address);
} finally {
lock.unlock();
}
}
D IN
NGE IONS
CHA CAT
128 4 LO
Crosscutting changes = aspect candidates

29. Contributions of the thesis
Fine-grained analysis of version archives. 1
Project-specific usage patterns of methods (FSE 2005)
Identification of cross-cutting changes (ASE 2006)
Mining bug databases to predict defects. 2
Dependencies predict defects (ISSRE 2007, ICSE 2008)
Domino effect: depending on defect-prone binaries increases
the chances of having defects (In submission).

30. Bugs! Bugs! Bugs!

31. Quality assurance is limited...
...by time...

32. Quality assurance is limited...
...by time... ...and by money.

33. Spent resources on the
components that need it most,
i.e., are most likely to fail.

34. Indicators of defects
• Code complexity
- Basili et al. 1996, Subramanyam and Krishnan 2003,
- Binkley and Schach 1998, Ohlsson and Alberg 1996,
- Nagappan et al. 2006, Knab et al. 2006

35. Indicators of defects
• Code complexity
- Basili et al. 1996, Subramanyam and Krishnan 2003,
- Binkley and Schach 1998, Ohlsson and Alberg 1996,
- Nagappan et al. 2006, Knab et al. 2006
• Code churn
- Nagappan and Ball 2005

36. Indicators of defects
• Code complexity
- Basili et al. 1996, Subramanyam and Krishnan 2003,
- Binkley and Schach 1998, Ohlsson and Alberg 1996,
- Nagappan et al. 2006, Knab et al. 2006
• Code churn
- Nagappan and Ball 2005
• Historical data
- Khoshgoftaar et al. 1996, Graves et al. 2000, Kim et al. 2007,
- Ostrand et al. 2005, Mockus et al. 2005

37. Indicators of defects
• Code complexity
- Basili et al. 1996, Subramanyam and Krishnan 2003,
- Binkley and Schach 1998, Ohlsson and Alberg 1996,
- Nagappan et al. 2006, Knab et al. 2006
• Code churn
- Nagappan and Ball 2005
• Historical data
- Khoshgoftaar et al. 1996, Graves et al. 2000, Kim et al. 2007,
- Ostrand et al. 2005, Mockus et al. 2005
• Code dependencies
- Nagappan and Ball 2007, Schröter et al. 2006

38. 2252 Binaries
28.3 MLOC

39. Windows Server layout

40. Windows Server layout

41. Windows Server layout

42. Windows Server layout

43. Hypotheses
Complexity of dependency graphs Sub
system
correlates with the number of post-release defects (H1) level
can predict the number of post-release defects (H2)

44. Hypotheses
Complexity of dependency graphs Sub
system
correlates with the number of post-release defects (H1) level
can predict the number of post-release defects (H2)
Network measures on dependency graphs Binary
correlate with the number of post-release defects (H3) level
can predict the number of post-release defects (H4)
can indicate critical “escrow” binaries (H5)

45. DATA. .

46. Data collection
Release point for
Windows Server 2003

47. Data collection
Release point for
Windows Server 2003
Dependencies
Network Measures
Complexity Metrics

48. Data collection
six months
Release point for
to collect
Windows Server 2003
defects
Dependencies
Network Measures
Complexity Metrics Defects

49. Dependencies
• Directed relationship between two pieces
of code (here: binaries)
• MaX dependency analysis framework
-Caller-callee dependencies
- Imports and exports
- RPC, COM
- Runtime dependencies (such as LoadLibrary)
- Registry access
- etc.

50. Centrality

51. Centrality
Degree
Blue binary has dependencies
to many other binaries

52. Centrality
Degree Closeness
Blue binary has dependencies Blue binary is close to all other
to many other binaries binaries (only two steps)

53. Centrality
Degree Closeness Betweenness
Blue binary has dependencies Blue binary is close to all other Blue binary connects the left
to many other binaries binaries (only two steps) with the right graph (bridge)

54. Centrality
• Degreethe number dependencies
centrality
-
counts
• Closeness centrality binaries into account
-
takes distance to all other
- Closeness: How close are the other binaries?
- Reach: How many binaries can be reached (weighted)?
- Eigenvector: similar to Pagerank
• Betweenness centrality paths through a binary
-
counts the number of shortest

55. Ego networks
EGO

56. Ego networks
EGO
INOUT

57. Ego networks
EGO
IN
INOUT

58. Ego networks
EGO
IN OUT
INOUT

59. Complexity metrics
Group Metrics Aggregation
Module metrics # functions in B
for a binary B # global variables in B
# executable lines in f()
# parameters in f()
Per-function metrics Total
# functions calling f()
for a function f() Max
# functions called by f()
McCabe’s cyclomatic complexity of f()
# methods in C
# subclasses of C
OO metrics Total
Depth of C in the inheritance tree
for a class C Max
Coupling between classes
Cyclic coupling between classes

60. RESULTS. .

61. 1 PATTERNS

62. Star pattern

63. Star pattern
With defects
No defects

64. Undirected cliques
... ...

65. Undirected cliques

66. Undirected cliques
Average number of defects is
higher for binaries in large cliques.

67. 2 PREDICTION

68. Prediction
Input metrics and measures Model Prediction
PCA
Regression

69. Prediction
Input metrics and measures Model Prediction
PCA
Regression
Metrics
SNA
Metrics+SNA

70. Prediction
Input metrics and measures Model Prediction
PCA
Regression
Metrics Classification
SNA
Metrics+SNA Ranking

71. Classification
Has a binary a defect or not?
or

72. Ranking
Which binaries have the most defects?
or or ... or

73. Random splits

74. Random splits
4×50×

75. Classification
(logistic regression)

76. Classification
(logistic regression)

77. Classification
(logistic regression)
SNA increases the recall by 0.10 (at p=0.01)
while precision remains comparable.

78. Ranking
(linear regression)

79. Ranking
(linear regression)
SNA+METRICS increases the correlation
by 0.10 (significant at p=0.01)

80. FUTURE WORK .
additions analysis architecture archives aspects bug cached calls
changes collaboration complexities component concerns cross-
cutting cvs data defects design development drawing dynamine
eclipse effort evolves failures fine-grained fix fix-inducing
graphs hatari history locate matching method mining
predicting program programmers report repositories
revision software support system taking transactions
version visualizing

81. FUTURE WORK .
analysis archives aspects bug changes collaboration
complexities component concerns cross-cutting cvs data defects
design development drawing eclipse erose evolves factor
failures fine-grained fix fix-inducing fm graphs guide hatari
history human matching mining networking
predicting program programmers quality report repositories
revision social software support system taking version

82. Collaboration
Comm. Version Bug
Archive Archive Database

83. Collaboration
Collab.
Data
Comm. Version Bug
Archive Archive Database

84. Collaboration
Collab. Effort
Data Data
Comm. Version Bug
Archive Archive Database

85. Collaboration
Collab. Effort
Data Data
Comm. Version Bug
Archive Archive Database

86. Collaboration
Collab. Effort
Data Data
Comm. Version Bug
Archive Archive Database
Social Networking for Software Development

87. Contributions of the thesis
Fine-grained analysis of version archives. 1
Project-specific usage patterns of methods (FSE 2005)
Identification of cross-cutting changes (ASE 2006)
Mining bug databases to predict defects. 2
Dependencies predict defects (ISSRE 2007, ICSE 2008)
Domino effect: depending on defect-prone binaries increases
the chances of having defects (In submission).