MINING
                                             SOFTW
                                                  ARE AR
       ...
Software development



         Build
Collaboration
Collaboration




Comm.
Archive
Collaboration




Comm.     Version
Archive   Archive
Collaboration




Comm.     Version     Bug
Archive   Archive   Database
Collaboration




Comm.      Version      Bug
Archive    Archive    Database


  Mining Software Archives
eROSE: Guiding developers

       Customers who
     bought this item also
          bought...



Purchase
 History
eROSE: Guiding developers

       Customers who               Developers who
     bought this item also       changed this...
eROSE suggests further locations.
THIS THESIS   .
THIS THESIS                                                           .
additions analysis architecture archives aspects  ...
THIS THESIS                                                           .
additions analysis architecture archives aspects  ...
Contributions of the thesis

Fine-grained analysis of version archives.              1
Project-specific usage patterns of ...
Contributions of the thesis

Fine-grained analysis of version archives.              1
Project-specific usage patterns of ...
Fine-grained analysis

public void createPartControl(Composite parent) {
    ...
    // add listener for editor page activ...
Fine-grained analysis

public void createPartControl(Composite parent) {
    ...
    // add listener for editor page activ...
Fine-grained analysis

public void createPartControl(Composite parent) {
    ...
    // add listener for editor page activ...
Fine-grained analysis

public void createPartControl(Composite parent) {
    ...
    // add listener for editor page activ...
Fine-grained analysis

public void createPartControl(Composite parent) {
    ...                                          ...
Fine-grained analysis

public void createPartControl(Composite parent) {
    ...                                          ...
Fine-grained analysis
Fine-grained analysis
         public static final native void _XFree(int address);
         public static final void XFree(...
Fine-grained analysis
         public static final native void _XFree(int address);
         public static final void XFree(...
Fine-grained analysis
                  public static final native void _XFree(int address);
                  public stati...
Contributions of the thesis

Fine-grained analysis of version archives.              1
Project-specific usage patterns of ...
Bugs! Bugs! Bugs!
Quality assurance is limited...

   ...by time...
Quality assurance is limited...

   ...by time...   ...and by money.
Spent resources on the
components that need it most,
  i.e., are most likely to fail.
Indicators of defects
•   Code complexity
    -   Basili et al. 1996, Subramanyam and Krishnan 2003,
    -   Binkley and S...
Indicators of defects
•   Code complexity
    -   Basili et al. 1996, Subramanyam and Krishnan 2003,
    -   Binkley and S...
Indicators of defects
•   Code complexity
    -   Basili et al. 1996, Subramanyam and Krishnan 2003,
    -   Binkley and S...
Indicators of defects
•   Code complexity
    -   Basili et al. 1996, Subramanyam and Krishnan 2003,
    -   Binkley and S...
2252 Binaries
28.3 MLOC
Windows Server layout
Windows Server layout
Windows Server layout
Windows Server layout
Hypotheses

Complexity of dependency graphs                             Sub
                                              ...
Hypotheses

Complexity of dependency graphs                             Sub
                                              ...
DATA.   .
Data collection

 Release point for
Windows Server 2003
Data collection

 Release point for
Windows Server 2003




  Dependencies

Network Measures

Complexity Metrics
Data collection
                      six months
 Release point for
                       to collect
Windows Server 2003
...
Dependencies
• Directed relationship between two pieces
  of code (here: binaries)
• MaX dependency analysis framework
  -...
Centrality
Centrality




Degree
Blue binary has dependencies
to many other binaries
Centrality




Degree                         Closeness
Blue binary has dependencies   Blue binary is close to all other
t...
Centrality




Degree                         Closeness                           Betweenness
Blue binary has dependencies...
Centrality
• Degreethe number dependencies
          centrality
   -
   counts

• Closeness centrality binaries into accou...
Ego networks




    EGO
Ego networks




    EGO




   INOUT
Ego networks




     EGO




IN
     INOUT
Ego networks




     EGO




IN           OUT
     INOUT
Complexity metrics
Group                  Metrics                                 Aggregation
Module metrics         # fun...
RESULTS.   .
1 PATTERNS
Star pattern
Star pattern

     With defects




               No defects
Undirected cliques



           ...       ...
Undirected cliques
Undirected cliques




    Average number of defects is
 higher for binaries in large cliques.
2 PREDICTION
Prediction


Input metrics and measures   Model        Prediction
                               PCA
                     ...
Prediction


Input metrics and measures   Model        Prediction
                               PCA
                     ...
Prediction


Input metrics and measures   Model        Prediction
                               PCA
                     ...
Classification


Has a binary a defect or not?




            or
Ranking


Which binaries have the most defects?




    or                or ... or
Random splits
Random splits




4×50×
Classification
 (logistic regression)
Classification
 (logistic regression)
Classification
            (logistic regression)




SNA increases the recall by 0.10 (at p=0.01)
  while precision remains...
Ranking
(linear regression)
Ranking
          (linear regression)




SNA+METRICS increases the correlation
    by 0.10 (significant at p=0.01)
FUTURE WORK                                                           .
additions analysis architecture archives aspects  ...
FUTURE WORK                                                             .
analysis archives aspects bug   changes collabor...
Collaboration




Comm.     Version     Bug
Archive   Archive   Database
Collaboration



          Collab.
           Data


Comm.               Version     Bug
Archive             Archive   Dat...
Collaboration



          Collab.             Effort
           Data               Data


Comm.               Version    ...
Collaboration



          Collab.             Effort
           Data               Data


Comm.               Version    ...
Collaboration



                  Collab.             Effort
                   Data               Data


        Comm.  ...
Contributions of the thesis

Fine-grained analysis of version archives.              1
Project-specific usage patterns of ...
Changes and Bugs: Mining and Predicting Development Activities
Changes and Bugs: Mining and Predicting Development Activities
Changes and Bugs: Mining and Predicting Development Activities
Upcoming SlideShare
Loading in...5
×

Changes and Bugs: Mining and Predicting Development Activities

2,175

Published on

PhD defense talk, 2008.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,175
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
1
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "Changes and Bugs: Mining and Predicting Development Activities"

  1. 1. MINING SOFTW ARE AR CHIVES Changes and Bugs Mining and Predicting Development Activities Thomas Zimmermann Saarbrücken, May 26, 2008
  2. 2. Software development Build
  3. 3. Collaboration
  4. 4. Collaboration Comm. Archive
  5. 5. Collaboration Comm. Version Archive Archive
  6. 6. Collaboration Comm. Version Bug Archive Archive Database
  7. 7. Collaboration Comm. Version Bug Archive Archive Database Mining Software Archives
  8. 8. eROSE: Guiding developers Customers who bought this item also bought... Purchase History
  9. 9. eROSE: Guiding developers Customers who Developers who bought this item also changed this function bought... also changed... Purchase Version History Archive
  10. 10. eROSE suggests further locations.
  11. 11. THIS THESIS .
  12. 12. THIS THESIS . additions analysis architecture archives aspects bug cached calls changes collaboration complexities component concerns cross- cutting cvs data defects design development drawing dynamine eclipse effort evolves failures fine-grained fix fix-inducing graphs hatari history locate matching method mining predicting program programmers report repositories revision software support system taking transactions version visualizing
  13. 13. THIS THESIS . additions analysis architecture archives aspects bug cached calls changes collaboration complexities component concerns cross- cutting cvs data defects design development drawing dynamine eclipse effort evolves failures fine-grained fix fix-inducing graphs hatari history locate matching method mining predicting program programmers report repositories revision software support system taking transactions version visualizing
  14. 14. Contributions of the thesis Fine-grained analysis of version archives. 1 Project-specific usage patterns of methods (FSE 2005) Identification of cross-cutting changes (ASE 2006) Mining bug databases to predict defects. 2 Dependencies predict defects (ISSRE 2007, ICSE 2008) Domino effect: depending on defect-prone binaries increases the chances of having defects (In submission).
  15. 15. Contributions of the thesis Fine-grained analysis of version archives. 1 Project-specific usage patterns of methods (FSE 2005) Identification of cross-cutting changes (ASE 2006) Mining bug databases to predict defects. 2 Dependencies predict defects (ISSRE 2007, ICSE 2008) Domino effect: depending on defect-prone binaries increases the chances of having defects (In submission).
  16. 16. Fine-grained analysis public void createPartControl(Composite parent) { ... // add listener for editor page activation getSite().getPage().addPartListener(partListener); } public void dispose() { ... getSite().getPage().removePartListener(partListener); }
  17. 17. Fine-grained analysis public void createPartControl(Composite parent) { ... // add listener for editor page activation getSite().getPage().addPartListener(partListener); } public void dispose() { ... getSite().getPage().removePartListener(partListener); }
  18. 18. Fine-grained analysis public void createPartControl(Composite parent) { ... // add listener for editor page activation getSite().getPage().addPartListener(partListener); } public void dispose() { co-added ... getSite().getPage().removePartListener(partListener); }
  19. 19. Fine-grained analysis public void createPartControl(Composite parent) { ... // add listener for editor page activation getSite().getPage().addPartListener(partListener); } public void dispose() { co-added ... getSite().getPage().removePartListener(partListener); }
  20. 20. Fine-grained analysis public void createPartControl(Composite parent) { ... close // add listener for editor page activation open getSite().getPage().addPartListener(partListener); println } public void dispose() { co-added ... getSite().getPage().removePartListener(partListener); } begin
  21. 21. Fine-grained analysis public void createPartControl(Composite parent) { ... close // add listener for editor page activation open getSite().getPage().addPartListener(partListener); println } public void dispose() { co-added ... getSite().getPage().removePartListener(partListener); } begin Co-added items = patterns
  22. 22. Fine-grained analysis
  23. 23. Fine-grained analysis public static final native void _XFree(int address); public static final void XFree(int /*long*/ address) { lock.lock(); try { _XFree(address); } finally { lock.unlock(); } }
  24. 24. Fine-grained analysis public static final native void _XFree(int address); public static final void XFree(int /*long*/ address) { lock.lock(); try { _XFree(address); } finally { lock.unlock(); } } D IN NGE IONS CHA CAT 128 4 LO
  25. 25. Fine-grained analysis public static final native void _XFree(int address); public static final void XFree(int /*long*/ address) { lock.lock(); try { _XFree(address); } finally { lock.unlock(); } } D IN NGE IONS CHA CAT 128 4 LO Crosscutting changes = aspect candidates
  26. 26. Contributions of the thesis Fine-grained analysis of version archives. 1 Project-specific usage patterns of methods (FSE 2005) Identification of cross-cutting changes (ASE 2006) Mining bug databases to predict defects. 2 Dependencies predict defects (ISSRE 2007, ICSE 2008) Domino effect: depending on defect-prone binaries increases the chances of having defects (In submission).
  27. 27. Bugs! Bugs! Bugs!
  28. 28. Quality assurance is limited... ...by time...
  29. 29. Quality assurance is limited... ...by time... ...and by money.
  30. 30. Spent resources on the components that need it most, i.e., are most likely to fail.
  31. 31. Indicators of defects • Code complexity - Basili et al. 1996, Subramanyam and Krishnan 2003, - Binkley and Schach 1998, Ohlsson and Alberg 1996, - Nagappan et al. 2006, Knab et al. 2006
  32. 32. Indicators of defects • Code complexity - Basili et al. 1996, Subramanyam and Krishnan 2003, - Binkley and Schach 1998, Ohlsson and Alberg 1996, - Nagappan et al. 2006, Knab et al. 2006 • Code churn - Nagappan and Ball 2005
  33. 33. Indicators of defects • Code complexity - Basili et al. 1996, Subramanyam and Krishnan 2003, - Binkley and Schach 1998, Ohlsson and Alberg 1996, - Nagappan et al. 2006, Knab et al. 2006 • Code churn - Nagappan and Ball 2005 • Historical data - Khoshgoftaar et al. 1996, Graves et al. 2000, Kim et al. 2007, - Ostrand et al. 2005, Mockus et al. 2005
  34. 34. Indicators of defects • Code complexity - Basili et al. 1996, Subramanyam and Krishnan 2003, - Binkley and Schach 1998, Ohlsson and Alberg 1996, - Nagappan et al. 2006, Knab et al. 2006 • Code churn - Nagappan and Ball 2005 • Historical data - Khoshgoftaar et al. 1996, Graves et al. 2000, Kim et al. 2007, - Ostrand et al. 2005, Mockus et al. 2005 • Code dependencies - Nagappan and Ball 2007, Schröter et al. 2006
  35. 35. 2252 Binaries 28.3 MLOC
  36. 36. Windows Server layout
  37. 37. Windows Server layout
  38. 38. Windows Server layout
  39. 39. Windows Server layout
  40. 40. Hypotheses Complexity of dependency graphs Sub system correlates with the number of post-release defects (H1) level can predict the number of post-release defects (H2)
  41. 41. Hypotheses Complexity of dependency graphs Sub system correlates with the number of post-release defects (H1) level can predict the number of post-release defects (H2) Network measures on dependency graphs Binary correlate with the number of post-release defects (H3) level can predict the number of post-release defects (H4) can indicate critical “escrow” binaries (H5)
  42. 42. DATA. .
  43. 43. Data collection Release point for Windows Server 2003
  44. 44. Data collection Release point for Windows Server 2003 Dependencies Network Measures Complexity Metrics
  45. 45. Data collection six months Release point for to collect Windows Server 2003 defects Dependencies Network Measures Complexity Metrics Defects
  46. 46. Dependencies • Directed relationship between two pieces of code (here: binaries) • MaX dependency analysis framework -Caller-callee dependencies - Imports and exports - RPC, COM - Runtime dependencies (such as LoadLibrary) - Registry access - etc.
  47. 47. Centrality
  48. 48. Centrality Degree Blue binary has dependencies to many other binaries
  49. 49. Centrality Degree Closeness Blue binary has dependencies Blue binary is close to all other to many other binaries binaries (only two steps)
  50. 50. Centrality Degree Closeness Betweenness Blue binary has dependencies Blue binary is close to all other Blue binary connects the left to many other binaries binaries (only two steps) with the right graph (bridge)
  51. 51. Centrality • Degreethe number dependencies centrality - counts • Closeness centrality binaries into account - takes distance to all other - Closeness: How close are the other binaries? - Reach: How many binaries can be reached (weighted)? - Eigenvector: similar to Pagerank • Betweenness centrality paths through a binary - counts the number of shortest
  52. 52. Ego networks EGO
  53. 53. Ego networks EGO INOUT
  54. 54. Ego networks EGO IN INOUT
  55. 55. Ego networks EGO IN OUT INOUT
  56. 56. Complexity metrics Group Metrics Aggregation Module metrics # functions in B for a binary B # global variables in B # executable lines in f() # parameters in f() Per-function metrics Total # functions calling f() for a function f() Max # functions called by f() McCabe’s cyclomatic complexity of f() # methods in C # subclasses of C OO metrics Total Depth of C in the inheritance tree for a class C Max Coupling between classes Cyclic coupling between classes
  57. 57. RESULTS. .
  58. 58. 1 PATTERNS
  59. 59. Star pattern
  60. 60. Star pattern With defects No defects
  61. 61. Undirected cliques ... ...
  62. 62. Undirected cliques
  63. 63. Undirected cliques Average number of defects is higher for binaries in large cliques.
  64. 64. 2 PREDICTION
  65. 65. Prediction Input metrics and measures Model Prediction PCA Regression
  66. 66. Prediction Input metrics and measures Model Prediction PCA Regression Metrics SNA Metrics+SNA
  67. 67. Prediction Input metrics and measures Model Prediction PCA Regression Metrics Classification SNA Metrics+SNA Ranking
  68. 68. Classification Has a binary a defect or not? or
  69. 69. Ranking Which binaries have the most defects? or or ... or
  70. 70. Random splits
  71. 71. Random splits 4×50×
  72. 72. Classification (logistic regression)
  73. 73. Classification (logistic regression)
  74. 74. Classification (logistic regression) SNA increases the recall by 0.10 (at p=0.01) while precision remains comparable.
  75. 75. Ranking (linear regression)
  76. 76. Ranking (linear regression) SNA+METRICS increases the correlation by 0.10 (significant at p=0.01)
  77. 77. FUTURE WORK . additions analysis architecture archives aspects bug cached calls changes collaboration complexities component concerns cross- cutting cvs data defects design development drawing dynamine eclipse effort evolves failures fine-grained fix fix-inducing graphs hatari history locate matching method mining predicting program programmers report repositories revision software support system taking transactions version visualizing
  78. 78. FUTURE WORK . analysis archives aspects bug changes collaboration complexities component concerns cross-cutting cvs data defects design development drawing eclipse erose evolves factor failures fine-grained fix fix-inducing fm graphs guide hatari history human matching mining networking predicting program programmers quality report repositories revision social software support system taking version
  79. 79. Collaboration Comm. Version Bug Archive Archive Database
  80. 80. Collaboration Collab. Data Comm. Version Bug Archive Archive Database
  81. 81. Collaboration Collab. Effort Data Data Comm. Version Bug Archive Archive Database
  82. 82. Collaboration Collab. Effort Data Data Comm. Version Bug Archive Archive Database
  83. 83. Collaboration Collab. Effort Data Data Comm. Version Bug Archive Archive Database Social Networking for Software Development
  84. 84. Contributions of the thesis Fine-grained analysis of version archives. 1 Project-specific usage patterns of methods (FSE 2005) Identification of cross-cutting changes (ASE 2006) Mining bug databases to predict defects. 2 Dependencies predict defects (ISSRE 2007, ICSE 2008) Domino effect: depending on defect-prone binaries increases the chances of having defects (In submission).

×