How Shallow is a Bug - How Open Source Communities (Help) Fix Bugs

Ranking the Bugs How Open Source Communities (Help) Fix Software Defects Diederik van Liere University of Toronto / Erasmus University Rotterdam FSOSS SENECA 2009 1

Who am I? • Post-doc researcher at the strategy department Rotman School of Management / University of Toronto • PhD information & decision sciences department at the Rotterdam School of Management • “Networkophile” • My research focuses on the intersection of social & digital networks and open source software • Mosaic / Netscape / Firefox user since 1994 • Blog: www.network-labs.org 2

My Coding Credentials Python: print “Hello FSOSS 2009” PHP: print “Hello FSOSS 2009”; jQuery: $(document).ready(function(){ $(“body”).append(“<p>Hello FSOSS 2009</ p>”); }); 3

So…it feels like being thrown in front of the lions 4

“All Bugs are Shallow Given Enough Eyeballs” 5 Linus Torvalds as quoted by Eric S. Raymond

But where is the empirical evidence? 6

Physical distance does not affect post-release fault rates, distance in the organizational chart does as Quoted from Greg Wilson, StackOveﬂow Days (Nagappan et al. (2007) & Bird et al. (2009)) 7

Data Collection • A community member is someone who posted at least 1 message at bugzilla.mozilla.org • Date of entry: ﬁrst time message posted • Date of exit: a month after last message posted • Collected bug reports ﬁled at bugzilla.mozilla.org with id 1 - 480.000 if product is Firefox / Core / Seamonkey, in total 320.655 bug reports • This covers the period late 1998 to March 1st, 2009 • Tried to match different email addresses to single developer (more about this later) 8

Starting point: a single bug report 14

Bug complexity Quality user contribution Time needed to Time needed to Community churn verify bug report ﬁx bug Understanding of the information repository 15

Bug complexity Quality user contribution Time needed to Time needed to Community churn verify bug report ﬁx bug Understanding of the information repository Stage 1 16

Bug complexity Quality user contribution Time needed to Time needed to Community churn verify bug report ﬁx bug Understanding of the information repository Stage 2 17

Estimation & Variables • Weibull regression (Accelerated Time to Failure models, used to predict time-to-failure for hard disks) • Unit of analysis: bug report • Quality of bug report ranges from 0 to 4: • Sum the presence of ‘steps to reproduce’, ‘stack trace’, ‘screenshot’ & ‘version information’ • Understanding of information repository: average experience bug discussants marking bugs duplicate • Churn rate community: • Bug complexity: centrality in bug dependency network 18

High quality contributions shorten time to verify Quality user Less time needed to contribution verify bug report 19

An open source community is also an information repository 20

Community members build, discuss & update the information repository 21

Understanding what’s in the information repository shortens time to verify Understanding of Less time needed to the information verify bug report repository 22

Community churn reduces understanding of the information repository More time needed to Community churn verify bug report 23

The Bigger Picture of Bug Complexity More time needed to Bug complexity ﬁx bug 24

Implications • Retention of community members is key • Get community members through the learning curve asap • Extend Bugzilla with prediction functionality to assist in allocating resources • Funnel 1st time bug reporters to ‘user assistance area’ 26

Tools to improve bug fixing process: 1. Predict Bugfix (Jetpack Add-on) 2. Crowdsource ‘flaming’ reports (Jetpack Add-on) 3. Crowdsource developer handles available at: www.network-labs.org 27

Predicting bug ﬁxes 28

Crowdsourcing ﬂamy bug reports 29

www.network-labs.org/mozilla/ 30

www.network-labs.org diederik.vanliere@rotman.utoronto.ca drdee_is_wired 32

Literature References • E. S. Raymond, “The Cathedral & the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary”, Sebastopol, CA: O'Reilly & Associates, Inc., 1999. Available at: http://www.catb.org/~esr/writings/cathedral- bazaar/ • N. Nagappan, et al., “The Inﬂuence of Organizational Structure on Software Quality - An Empirical Case Study”, International Conference on Software Engineering, 2008. Available at: http://portal.acm.org/citation.cfm? id=1368160 • T. Zimmermann, et al. “Predicting Defects Using Network Analysis on Dependency Graphs”, International Conference on Software Engineering, 2008. Available at: http://portal.acm.org/citation.cfm?id=1368161 • C. Bird, et al. “Does Distributed Development Affect Software Quality? An Empirical Case Study of Windows Vista”, Communications of the ACM, 2009. Available at: http://macbeth.cs.ucdavis.edu/distributed.pdf 33

References • http://www.flickr.com/photos/eriknielsen/2233837359/ sizes/l/ • http://www.flickr.com/photos/fastjack/282707058/sizes/o/ • http://www.flickr.com/photos/ 59303791@N00/3398708956/sizes/o/ • http://www.flickr.com/photos/_ilkin_/3890567460/sizes/o/ • http://www.flickr.com/photos/wondertubs/2152108411/ • http://www.flickr.com/photos/asurroca/136223817/sizes/o/ • http://www.flickr.com/photos/thinmints137/452928157/ sizes/o/ 34

References • http://www.flickr.com/photos/hanneorla/4032004209/sizes/o/ • http://www.flickr.com/photos/97629199@N00/3254201501/in/pool- bookshelf • http://www.flickr.com/photos/gadl/740994053/ • http://www.flickr.com/photos/blu_blue/262096844/sizes/o/ • http://www.flickr.com/photos/kurisuuu/3227744533/sizes/o/ • http://www.flickr.com/photos/astrid/2330867426/sizes/o/ • http://www.flickr.com/photos/sweetone/2666516868/sizes/o/ • http://www.flickr.com/photos/caveman_92223/2982595969/sizes/o/ • http://www.flickr.com/photos/mrtea/2114891329/sizes/o/ 35

http://ultimategerardm.blogspot.com	334
http://ultimategerardm.blogspot.in	24
http://www.slideshare.net	14
http://ultimategerardm.blogspot.com.au	7
http://paper.li	5
http://ultimategerardm.blogspot.fr	4
http://ultimategerardm.blogspot.ca	3
http://ultimategerardm.blogspot.it	2
http://static.slidesharecdn.com	1
http://www.lmodules.com	1
http://ultimategerardm.blogspot.se	1
http://ultimategerardm.blogspot.co.uk	1

How Shallow is a Bug - How Open Source Communities (Help) Fix Bugs

by Diederik van Liere on Oct 30, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

12 Embeds 397

Statistics

How Shallow is a Bug - How Open Source Communities (Help) Fix Bugs — Presentation Transcript