Ranking the Bugs
How Open Source Communities
(Help) Fix Software Defects
Diederik van Liere
University of Toronto / Erasmus University Rotterdam
FSOSS SENECA 2009
1

Who am I?
• Post-doc researcher at the strategy department
Rotman School of Management / University of
Toronto
• PhD information & decision sciences department at
the Rotterdam School of Management
• “Networkophile”
• My research focuses on the intersection of social &
digital networks and open source software
• Mosaic / Netscape / Firefox user since 1994
• Blog: www.network-labs.org
2

My Coding Credentials
Python:
print “Hello FSOSS 2009”
PHP:
print “Hello FSOSS 2009”;
jQuery:
$(document).ready(function(){
$(“body”).append(“<p>Hello FSOSS 2009</
p>”);
}); 3

So…it feels like being thrown in front of the lions
4

“All Bugs are Shallow Given
Enough Eyeballs”
5 Linus Torvalds as quoted by Eric S. Raymond

6

But where is the empirical evidence?
6

Physical distance does not affect
post-release fault rates, distance in
the organizational chart does
as Quoted from Greg Wilson, StackOveflow Days (Nagappan et al. (2007) & Bird et al. (2009))
7

Data Collection
• A community member is someone who posted at
least 1 message at bugzilla.mozilla.org
• Date of entry: first time message posted
• Date of exit: a month after last message posted
• Collected bug reports filed at bugzilla.mozilla.org
with id 1 - 480.000 if product is Firefox / Core /
Seamonkey, in total 320.655 bug reports
• This covers the period late 1998 to March 1st, 2009
• Tried to match different email addresses to single
developer (more about this later)
8

9

10

11

12

13

Starting point:
a single bug report
14

Bug complexity
Quality user
contribution
Time needed to Time needed to
Community churn
verify bug report fix bug
Understanding of
the information
repository
15

Bug complexity
Quality user
contribution
Time needed to Time needed to
Community churn
verify bug report fix bug
Understanding of
the information
repository
Stage 1
16

Bug complexity
Quality user
contribution
Time needed to Time needed to
Community churn
verify bug report fix bug
Understanding of
the information
repository
Stage 2
17

Estimation & Variables
• Weibull regression (Accelerated Time to Failure
models, used to predict time-to-failure for hard disks)
• Unit of analysis: bug report
• Quality of bug report ranges from 0 to 4:
• Sum the presence of ‘steps to reproduce’, ‘stack
trace’, ‘screenshot’ & ‘version information’
• Understanding of information repository: average
experience bug discussants marking bugs duplicate
• Churn rate community:
• Bug complexity: centrality in bug dependency
network 18

High quality contributions
shorten time to verify
Quality user Less time needed to
contribution verify bug report
19

An open source community is also an
information repository
20

Community members
build, discuss & update the
information repository
21

Understanding what’s in the
information repository
shortens time to verify
Understanding of
Less time needed to
the information
verify bug report
repository
22

Community churn
reduces understanding of the
information repository
More time needed to
Community churn
verify bug report
23

The Bigger Picture of Bug Complexity
More time needed to
Bug complexity
fix bug
24

25

Implications
• Retention of community members is key
• Get community members through the learning
curve asap
• Extend Bugzilla with prediction functionality
to assist in allocating resources
• Funnel 1st time bug reporters to ‘user
assistance area’
26

Tools to improve bug
fixing process:
1. Predict Bugfix (Jetpack Add-on)
2. Crowdsource ‘flaming’ reports
(Jetpack Add-on)
3. Crowdsource developer
handles
available at:
www.network-labs.org
27

Predicting bug fixes
28

Crowdsourcing flamy bug
reports
29

www.network-labs.org/mozilla/
30

31

www.network-labs.org
diederik.vanliere@rotman.utoronto.ca
drdee_is_wired 32

Literature References
• E. S. Raymond, “The Cathedral & the Bazaar: Musings on Linux and Open
Source by an Accidental Revolutionary”, Sebastopol, CA: O'Reilly & Associates,
Inc., 1999. Available at: http://www.catb.org/~esr/writings/cathedral-
bazaar/
• N. Nagappan, et al., “The Influence of Organizational Structure on Software
Quality - An Empirical Case Study”, International Conference on Software
Engineering, 2008. Available at: http://portal.acm.org/citation.cfm?
id=1368160
• T. Zimmermann, et al. “Predicting Defects Using Network Analysis on Dependency
Graphs”, International Conference on Software Engineering, 2008.
Available at: http://portal.acm.org/citation.cfm?id=1368161
• C. Bird, et al. “Does Distributed Development Affect Software Quality? An Empirical
Case Study of Windows Vista”, Communications of the ACM, 2009.
Available at: http://macbeth.cs.ucdavis.edu/distributed.pdf
33

References
• http://www.flickr.com/photos/eriknielsen/2233837359/
sizes/l/
• http://www.flickr.com/photos/fastjack/282707058/sizes/o/
• http://www.flickr.com/photos/
59303791@N00/3398708956/sizes/o/
• http://www.flickr.com/photos/_ilkin_/3890567460/sizes/o/
• http://www.flickr.com/photos/wondertubs/2152108411/
• http://www.flickr.com/photos/asurroca/136223817/sizes/o/
• http://www.flickr.com/photos/thinmints137/452928157/
sizes/o/
34

References
• http://www.flickr.com/photos/hanneorla/4032004209/sizes/o/
• http://www.flickr.com/photos/97629199@N00/3254201501/in/pool-
bookshelf
• http://www.flickr.com/photos/gadl/740994053/
• http://www.flickr.com/photos/blu_blue/262096844/sizes/o/
• http://www.flickr.com/photos/kurisuuu/3227744533/sizes/o/
• http://www.flickr.com/photos/astrid/2330867426/sizes/o/
• http://www.flickr.com/photos/sweetone/2666516868/sizes/o/
• http://www.flickr.com/photos/caveman_92223/2982595969/sizes/o/
• http://www.flickr.com/photos/mrtea/2114891329/sizes/o/
35