Ranking the Bugs
    How Open Source Communities
      (Help) Fix Software Defects




                  Diederik van Lier...
Who am I?
•   Post-doc researcher at the strategy department
    Rotman School of Management / University of
    Toronto
•...
My Coding Credentials
Python:
   print “Hello FSOSS 2009”
PHP:
   print “Hello FSOSS 2009”;
jQuery:
   $(document).ready(f...
So…it feels like being thrown in front of the lions
                         4
“All Bugs are Shallow Given
      Enough Eyeballs”
             5    Linus Torvalds as quoted by Eric S. Raymond
6
But where is the empirical evidence?
                 6
Physical distance does not affect
post-release fault rates, distance in
  the organizational chart does
        as Quoted ...
Data Collection
•   A community member is someone who posted at
    least 1 message at bugzilla.mozilla.org
    •   Date o...
9
10
11
12
13
Starting point:
a single bug report
         14
Bug complexity



  Quality user
  contribution




                   Time needed to      Time needed to
Community churn
...
Bug complexity



  Quality user
  contribution




                    Time needed to      Time needed to
Community churn...
Bug complexity



  Quality user
  contribution




                   Time needed to       Time needed to
Community churn...
Estimation & Variables
•   Weibull regression (Accelerated Time to Failure
    models, used to predict time-to-failure for...
High quality contributions
shorten time to verify




                 Quality user   Less time needed to
                ...
An open source community is also an
information repository




               20
Community members
build, discuss & update the
information repository



             21
Understanding what’s in the
information repository
 shortens time to verify


                  Understanding of
         ...
Community churn
reduces understanding of the
information repository




                                    More time need...
The Bigger Picture of Bug Complexity




                                       More time needed to
                      ...
25
Implications
•   Retention of community members is key

•   Get community members through the learning
    curve asap

•  ...
Tools to improve bug
   fixing process:
1. Predict Bugfix (Jetpack Add-on)
2. Crowdsource ‘flaming’ reports
(Jetpack Add-on)
...
Predicting bug fixes




         28
Crowdsourcing flamy bug
       reports




          29
www.network-labs.org/mozilla/




              30
31
www.network-labs.org
diederik.vanliere@rotman.utoronto.ca

drdee_is_wired   32
Literature References
•   E. S. Raymond, “The Cathedral & the Bazaar: Musings on Linux and Open
    Source by an Accidenta...
References
•   http://www.flickr.com/photos/eriknielsen/2233837359/
    sizes/l/
•   http://www.flickr.com/photos/fastjack/2...
References
•   http://www.flickr.com/photos/hanneorla/4032004209/sizes/o/

•   http://www.flickr.com/photos/97629199@N00/325...
Upcoming SlideShare
Loading in …5
×

How Shallow is a Bug - How Open Source Communities (Help) Fix Bugs

2,965 views

Published on

An important question for managing open source communities is how to allocate the resources of volunteers among the many tasks. One time consuming task is reviewing new bug reports. My presentation consists of three parts: First, based on a detailed analysis over the period 1999 -2009 of the Firefox Bugzilla database I will present graphs showing the role of community members in fixing bugs over the period 1999 - 2009. Second, I will focus on the role of an open source community as an information repository, how such an information repository is build by community members, and how understanding of this artifact shortens repair times. Finally I will talk about some small tools that I developed to help improve the bug fixing process. One of these tools predicts which bug report will get fixed based only on the initial bug report information. My goal is to inform open source community members and give some empirical evidence which might help in adding new functionality to Bugzilla where new bugs are not just ranked based on submission date but are ranked based on most likely to be fixed.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,965
On SlideShare
0
From Embeds
0
Number of Embeds
712
Actions
Shares
0
Downloads
26
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • How Shallow is a Bug - How Open Source Communities (Help) Fix Bugs

    1. 1. Ranking the Bugs How Open Source Communities (Help) Fix Software Defects Diederik van Liere University of Toronto / Erasmus University Rotterdam FSOSS SENECA 2009 1
    2. 2. Who am I? • Post-doc researcher at the strategy department Rotman School of Management / University of Toronto • PhD information & decision sciences department at the Rotterdam School of Management • “Networkophile” • My research focuses on the intersection of social & digital networks and open source software • Mosaic / Netscape / Firefox user since 1994 • Blog: www.network-labs.org 2
    3. 3. My Coding Credentials Python: print “Hello FSOSS 2009” PHP: print “Hello FSOSS 2009”; jQuery: $(document).ready(function(){ $(“body”).append(“<p>Hello FSOSS 2009</ p>”); }); 3
    4. 4. So…it feels like being thrown in front of the lions 4
    5. 5. “All Bugs are Shallow Given Enough Eyeballs” 5 Linus Torvalds as quoted by Eric S. Raymond
    6. 6. 6
    7. 7. But where is the empirical evidence? 6
    8. 8. Physical distance does not affect post-release fault rates, distance in the organizational chart does as Quoted from Greg Wilson, StackOveflow Days (Nagappan et al. (2007) & Bird et al. (2009)) 7
    9. 9. Data Collection • A community member is someone who posted at least 1 message at bugzilla.mozilla.org • Date of entry: first time message posted • Date of exit: a month after last message posted • Collected bug reports filed at bugzilla.mozilla.org with id 1 - 480.000 if product is Firefox / Core / Seamonkey, in total 320.655 bug reports • This covers the period late 1998 to March 1st, 2009 • Tried to match different email addresses to single developer (more about this later) 8
    10. 10. 9
    11. 11. 10
    12. 12. 11
    13. 13. 12
    14. 14. 13
    15. 15. Starting point: a single bug report 14
    16. 16. Bug complexity Quality user contribution Time needed to Time needed to Community churn verify bug report fix bug Understanding of the information repository 15
    17. 17. Bug complexity Quality user contribution Time needed to Time needed to Community churn verify bug report fix bug Understanding of the information repository Stage 1 16
    18. 18. Bug complexity Quality user contribution Time needed to Time needed to Community churn verify bug report fix bug Understanding of the information repository Stage 2 17
    19. 19. Estimation & Variables • Weibull regression (Accelerated Time to Failure models, used to predict time-to-failure for hard disks) • Unit of analysis: bug report • Quality of bug report ranges from 0 to 4: • Sum the presence of ‘steps to reproduce’, ‘stack trace’, ‘screenshot’ & ‘version information’ • Understanding of information repository: average experience bug discussants marking bugs duplicate • Churn rate community: • Bug complexity: centrality in bug dependency network 18
    20. 20. High quality contributions shorten time to verify Quality user Less time needed to contribution verify bug report 19
    21. 21. An open source community is also an information repository 20
    22. 22. Community members build, discuss & update the information repository 21
    23. 23. Understanding what’s in the information repository shortens time to verify Understanding of Less time needed to the information verify bug report repository 22
    24. 24. Community churn reduces understanding of the information repository More time needed to Community churn verify bug report 23
    25. 25. The Bigger Picture of Bug Complexity More time needed to Bug complexity fix bug 24
    26. 26. 25
    27. 27. Implications • Retention of community members is key • Get community members through the learning curve asap • Extend Bugzilla with prediction functionality to assist in allocating resources • Funnel 1st time bug reporters to ‘user assistance area’ 26
    28. 28. Tools to improve bug fixing process: 1. Predict Bugfix (Jetpack Add-on) 2. Crowdsource ‘flaming’ reports (Jetpack Add-on) 3. Crowdsource developer handles available at: www.network-labs.org 27
    29. 29. Predicting bug fixes 28
    30. 30. Crowdsourcing flamy bug reports 29
    31. 31. www.network-labs.org/mozilla/ 30
    32. 32. 31
    33. 33. www.network-labs.org diederik.vanliere@rotman.utoronto.ca drdee_is_wired 32
    34. 34. Literature References • E. S. Raymond, “The Cathedral & the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary”, Sebastopol, CA: O'Reilly & Associates, Inc., 1999. Available at: http://www.catb.org/~esr/writings/cathedral- bazaar/ • N. Nagappan, et al., “The Influence of Organizational Structure on Software Quality - An Empirical Case Study”, International Conference on Software Engineering, 2008. Available at: http://portal.acm.org/citation.cfm? id=1368160 • T. Zimmermann, et al. “Predicting Defects Using Network Analysis on Dependency Graphs”, International Conference on Software Engineering, 2008. Available at: http://portal.acm.org/citation.cfm?id=1368161 • C. Bird, et al. “Does Distributed Development Affect Software Quality? An Empirical Case Study of Windows Vista”, Communications of the ACM, 2009. Available at: http://macbeth.cs.ucdavis.edu/distributed.pdf 33
    35. 35. References • http://www.flickr.com/photos/eriknielsen/2233837359/ sizes/l/ • http://www.flickr.com/photos/fastjack/282707058/sizes/o/ • http://www.flickr.com/photos/ 59303791@N00/3398708956/sizes/o/ • http://www.flickr.com/photos/_ilkin_/3890567460/sizes/o/ • http://www.flickr.com/photos/wondertubs/2152108411/ • http://www.flickr.com/photos/asurroca/136223817/sizes/o/ • http://www.flickr.com/photos/thinmints137/452928157/ sizes/o/ 34
    36. 36. References • http://www.flickr.com/photos/hanneorla/4032004209/sizes/o/ • http://www.flickr.com/photos/97629199@N00/3254201501/in/pool- bookshelf • http://www.flickr.com/photos/gadl/740994053/ • http://www.flickr.com/photos/blu_blue/262096844/sizes/o/ • http://www.flickr.com/photos/kurisuuu/3227744533/sizes/o/ • http://www.flickr.com/photos/astrid/2330867426/sizes/o/ • http://www.flickr.com/photos/sweetone/2666516868/sizes/o/ • http://www.flickr.com/photos/caveman_92223/2982595969/sizes/o/ • http://www.flickr.com/photos/mrtea/2114891329/sizes/o/ 35

    ×