Predicting Build Failures using Social Netowrok Analysis on Developer Communication

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    Notes on slide 1

    Favorites, Groups & Events

    Predicting Build Failures using Social Netowrok Analysis on Developer Communication - Presentation Transcript

    1. Predicting Build Failures using Social Network Analysis on Developer Communication Timo Wolf Adrian Schröter Daniela Damian Thanh Nguyen SIEMENS AG Software Engineering Global interAction Lab – SEGAL Corporate Technology Computer Science Department Software & Engineering University of Victoria, BC Canada
    2. Introduction SIEMENS AG timowolf@siemens.com 2
    3. Introduction SIEMENS AG timowolf@siemens.com 2
    4. Introduction Build Failure - Compile Error - Test Failure SIEMENS AG timowolf@siemens.com 2
    5. Introduction The damn build broke again! WHY ? Build Failure - Compile Error - Test Failure SIEMENS AG timowolf@siemens.com 2
    6. DB changed! Introduction The damn build broke again! WHY ? But that’s transparent. Analyses takes time! We should have talked earlier! Build Failure - Compile Error - Test Failure SIEMENS AG timowolf@siemens.com 2
    7. DB changed! Introduction The damn build broke again! WHY ? But that’s transparent. Analyses takes time! We should have talked earlier! Build Failure - Compile Error - Test Failure • Collaboration essential in software development • Communication problems lead to coordination and integration failures SIEMENS AG timowolf@siemens.com 2
    8. DB changed! Introduction The damn build broke again! WHY ? But that’s transparent. Analyses takes time! We should have talked earlier! Build Failure - Compile Error - Test Failure • Collaboration essential in software development • Communication problems lead to coordination and integration failures Do specific communication behaviors enable effective and successful coordination? SIEMENS AG timowolf@siemens.com 2
    9. Communication for collaboration SIEMENS AG timowolf@siemens.com 3
    10. Communication for collaboration Task related Content & Information • Communication: • Task related content SIEMENS AG timowolf@siemens.com 3
    11. Communication for collaboration Task related Content & Information • Communication: • Task related content • Structure SIEMENS AG timowolf@siemens.com 3
    12. Communication for collaboration • Communication: • Task related content • Structure Our focus: Communication Structure and relationship to coordination outcome SIEMENS AG timowolf@siemens.com 3
    13. Study Settings: IBM Jazz SIEMENS AG timowolf@siemens.com 4
    14. Study Settings: IBM Jazz & IBM Research SIEMENS AG timowolf@siemens.com 4
    15. Study Settings: IBM Jazz 151 developers 47 functional teams SIEMENS AG timowolf@siemens.com 5
    16. Study Settings: IBM Jazz 7 development sites in USA, Canada, Europe SIEMENS AG timowolf@siemens.com 6
    17. Study Settings: IBM Jazz SIEMENS AG timowolf@siemens.com 7
    18. Coordination & Integration in Jazz SIEMENS AG timowolf@siemens.com 8
    19. Coordination & Integration in Jazz Team A Contributor 1 Contributor 2 Contributor 3 Team A Stream Build 1 SIEMENS AG timowolf@siemens.com 8
    20. Coordination & Integration in Jazz Team A Contributor 1 Contributor 2 Contributor 3 Team A !! !! !! Stream Build 1 SIEMENS AG timowolf@siemens.com 8
    21. Coordination & Integration in Jazz Team A Contributor 1 Contributor 2 Contributor 3 Team A Stream Build 1 SIEMENS AG timowolf@siemens.com 8
    22. Coordination & Integration in Jazz Team A Contributor 1 Contributor 2 Contributor 3 Team A Stream Build 1 Build 1 Team B Stream Change commits changes Sets Team B Contributor 4 Contributor 5 Contributor 6 SIEMENS AG timowolf@siemens.com 8
    23. Coordination & Integration in Jazz Team A Contributor 1 Contributor 2 Contributor 3 Team A Stream Build 1 Project Project Build 1 Integration Stream get changes publish changes Build 1 Team B Stream Change commits changes Sets Team B Contributor 4 Contributor 5 Contributor 6 SIEMENS AG timowolf@siemens.com 8
    24. Coordination & Integration in Jazz Team A Contributor 1 Contributor 2 Contributor 3 Team A Stream Build 1 Project Project Build 1 Integration Stream get changes publish changes Build 1 Team B Stream Change commits changes Sets Team B Contributor 4 Contributor 5 Contributor 6 Coordination Outcome: Build Result SIEMENS AG timowolf@siemens.com 8
    25. Research Questions SIEMENS AG timowolf@siemens.com 9
    26. Research Questions ! OK ERROR SIEMENS AG timowolf@siemens.com 9
    27. Research Questions ! OK ERROR SNA Measures SNA Measures measure e.g. Density = 0.8 predict measure e.g. Density = 0.4 predict SIEMENS AG timowolf@siemens.com 9
    28. Research Questions ! OK ERROR SNA Measures SNA Measures measure e.g. Density = 0.8 predict measure e.g. Density = 0.4 predict • RQ1: Can individual measures of communication structure predict integration failure? SIEMENS AG timowolf@siemens.com 9
    29. Research Questions ! OK ERROR SNA Measures SNA Measures measure e.g. Density = 0.8 predict measure e.g. Density = 0.4 predict • RQ1: Can individual measures of communication structure predict integration failure? • RQ2: Can the combination of communication structure measures predict integration failure? SIEMENS AG timowolf@siemens.com 9
    30. Constructing Social Networks in Jazz SIEMENS AG timowolf@siemens.com 10
    31. Constructing Social Networks in Jazz Previous Previous Build Build N N Build Build FailedFailed Stream t0 t1 time SIEMENS AG timowolf@siemens.com 10
    32. Constructing Social Networks in Jazz c3 c1 c2 c4 Previous Previous Build N Network Build Build N N Build Build FailedFailed Stream t0 t1 time SIEMENS AG timowolf@siemens.com 10
    33. Constructing Social Networks in Jazz c3 c1 c2 c4 Previous Previous Build N Network Build Build N N Build Build FailedFailed Stream t0 t1 time c1 c2 c3 c4 SIEMENS AG timowolf@siemens.com 10
    34. Constructing Social Networks in Jazz c3 c1 c2 c4 Previous Previous Build N Network Build Build N N Build Build FailedFailed Stream t0 t1 time c1 c2 c3 c4 w1 w2 SIEMENS AG timowolf@siemens.com 10
    35. Constructing Social Networks in Jazz c3 c1 c2 c4 Previous Previous Build N Network Build Build N N Build Build FailedFailed Stream t0 t1 time c1 c2 c3 c4 w1 w2 SIEMENS AG timowolf@siemens.com 10
    36. Constructing Social Networks in Jazz Social Network Analysis (SNA) c3 quantitative c1 SNA measures of communication structure c2 c4 Previous Previous Build N Network Build Build N N Build Build FailedFailed Stream t0 t1 time c1 c2 c3 c4 w1 w2 SIEMENS AG timowolf@siemens.com 10
    37. SNA Measures SIEMENS AG timowolf@siemens.com 11
    38. SNA Measures SIEMENS AG timowolf@siemens.com 12
    39. SNA Measures SIEMENS AG timowolf@siemens.com 13
    40. SNA Measures SIEMENS AG timowolf@siemens.com 14
    41. SNA Measures SIEMENS AG timowolf@siemens.com 15
    42. Data Collection & Selection • Time Range: Nov. 5, 2007 - Feb. 26, 2008 • 1288 Build Results • 25713 Work Items • 71019 Comments • Selected Build Results of 5 Teams • Selected 3 Integration Builds (nightly, weekly, beta) SIEMENS AG timowolf@siemens.com 16
    43. RQ1: individual measure prediction? SIEMENS AG timowolf@siemens.com 17
    44. RQ1: individual measure prediction? SN Build 1 SN Build 2 SN Build 3 SN Build 5 SN Build 4 SN Build 6 SIEMENS AG timowolf@siemens.com 17
    45. RQ1: individual measure prediction? Social Network Measures • Density for Completeness • SN Build 1 SN Build 2 Degree Centrality for Activity SN Build 3 • Betweenness Centrality for Control SN Build 5 SN Build 4 • Structural Holes for Redundancy SN Build 6 SIEMENS AG timowolf@siemens.com 17
    46. RQ1: individual measure prediction? Social Network Measures • Density for Completeness • SN Build 1 SN Build 2 Degree Centrality for Activity SN Build 3 • Betweenness Centrality for Control SN Build 5 SN Build 4 • Structural Holes for Redundancy SN Build 6 No statistically significant difference! SIEMENS AG timowolf@siemens.com 17
    47. RQ2: multi measures prediction? SIEMENS AG timowolf@siemens.com 18
    48. RQ2: multi measures prediction? ? SIEMENS AG timowolf@siemens.com 18
    49. RQ2: multi measures prediction? ? 1. Train with all measures Bayesian Classifier SIEMENS AG timowolf@siemens.com 18
    50. RQ2: multi measures prediction? ? 1. Train with 2. Input all measures Bayesian Classifier SIEMENS AG timowolf@siemens.com 18
    51. RQ2: multi measures prediction? ? 1. Train with 2. Input all measures 3. Predict Bayesian Classifier SIEMENS AG timowolf@siemens.com 18
    52. RQ2: multi measures prediction? ? 1. Train with 2. Input all measures 3. Predict Bayesian Classifier SIEMENS AG timowolf@siemens.com 18
    53. Prediction Results SIEMENS AG timowolf@siemens.com 19
    54. Prediction Results ERROR Team Builds Project Builds B C F P W nightly weekly beta Recall 55% 75% 62% 66% 74% 89% 100% 92% Precision 52% 50% 75% 76% 66% 73% 92% 92% SIEMENS AG timowolf@siemens.com 19
    55. Prediction Results ERROR Team Builds Project Builds B C F P W nightly weekly beta Recall 55% 75% 62% 66% 74% 89% 100% 92% Precision 52% 50% 75% 76% 66% 73% 92% 92% SIEMENS AG timowolf@siemens.com 19
    56. Prediction Results ERROR Team Builds Project Builds B C F P W nightly weekly beta Recall 55% 75% 62% 66% 74% 89% 100% 92% Precision 52% 50% 75% 76% 66% 73% 92% 92% • Recall: • 62 of 100 failed builds are classified correctly SIEMENS AG timowolf@siemens.com 19
    57. Prediction Results ERROR Team Builds Project Builds B C F P W nightly weekly beta Recall 55% 75% 62% 66% 74% 89% 100% 92% Precision 52% 50% 75% 76% 66% 73% 92% 92% • Recall: • 62 of 100 failed builds are classified correctly • Precision: • 75 of 100 failure predictions are actual failures • 75% confidence that a prediction is correct SIEMENS AG timowolf@siemens.com 19
    58. Summary SIEMENS AG timowolf@siemens.com 20
    59. Summary • Quantitative empirical evidence: Communication structures make a difference! SIEMENS AG timowolf@siemens.com 20
    60. Summary • Quantitative empirical evidence: Communication structures make a difference! • Communication structures: • impact integration build results • are key for successful collaboration SIEMENS AG timowolf@siemens.com 20
    61. Summary • Quantitative empirical evidence: Communication structures make a difference! • Communication structures: • impact integration build results • are key for successful collaboration • Single measures do not predict build results SIEMENS AG timowolf@siemens.com 20
    62. Summary • Quantitative empirical evidence: Communication structures make a difference! • Communication structures: • impact integration build results • are key for successful collaboration • Single measures do not predict build results • Combination measures do predict build results SIEMENS AG timowolf@siemens.com 20
    63. Practical Implications • Communication assessment for future builds • Notification system for build failures • Communication structure improvement SIEMENS AG timowolf@siemens.com 21
    64. Future Work • Factor analysis of combined measures • Include technical information • Prediction effectiveness case study SIEMENS AG timowolf@siemens.com 22
    65. Questions? Timo Wolf Daniela Damian timowolf@siemens.com danielad@cs.uvic.ca Adrian Schröter Thanh Nguyen schadr@uvic.ca duythanh@uvic.ca SIEMENS AG Corporate Technology Software & Engineering http://segal.uvic.ca
    66. Backup Slides
    67. Descriptive Statistics I SIEMENS AG timowolf@siemens.com 25
    68. Descriptive Statistics II SIEMENS AG timowolf@siemens.com 26
    SlideShare Zeitgeist 2009

    + University of VictoriaUniversity of Victoria Nominate

    custom

    364 views, 0 favs, 0 embeds more stats

    This is a presentation given by Timo Wolf during th more

    More info about this document

    © All Rights Reserved

    Go to text version

    • Total Views 364
      • 364 on SlideShare
      • 0 from embeds
    • Comments 0
    • Favorites 0
    • Downloads 6
    Most viewed embeds

    more

    All embeds

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories

    Tags

    -->
    What's New

    © 2009 SlideShare Inc. All rights reserved.