Over the past decade, several research efforts have studied the quality of software systems by looking at post-release bugs. However, these studies do not account for bugs that remain dormant (i.e., introduced in a version of the software system, but are not found until much later) for years and across many versions. Such dormant bugs skew our under- standing of the software quality. In this paper we study dormant bugs against non-dormant bugs using data from 20 different open-source Apache foundation software systems. We find that 33% of the bugs introduced in a version are not reported till much later (i.e., they are reported in future versions as dormant bugs). Moreover, we find that 18.9% of the reported bugs in a version are not even introduced in that version (i.e., they are dormant bugs from prior versions). In short, the use of reported bugs to judge the quality of a specific version might be misleading. Exploring the fix process for dormant bugs, we find that they are fixed faster (median fix time of 5 days) than non- dormant bugs (median fix time of 8 days), and are fixed by more experienced developers (median commit counts of developers who fix dormant bug is 169% higher). Our results highlight that dormant bugs are different from non-dormant bugs in many perspectives and that future research in software quality should carefully study and consider dormant bugs.
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
MSR2014 - An Empirical Study of Dormant Bugs
1. An Empirical Study of Dormant Bugs
Tse-Hsun(Peter) Chen Meiyappan Nagappan Emad Shihab Ahmed E. Hassan
1
2. Most studies focus on post-release bugs
2
Version A Version B
File
A
Bug 1 Bug 2
Bugs Discovered!Assign Bugs to File A
3. Some bugs may remain dormant for a
long time
3
Version A Version CVersion B
File
A
Bug is waken up, but is
counted as post-release
bug for Version B
Zzz… Zzz…
4. What are dormant bugs?
4
Version BVersion A Version C Today
File
A
Bug 1
File B
Bug 2 Bug 3
Bugs
introduced in
Version A
Not a
dormant bug
Bug 2
Dormant bug
(discovered one
version after it
was introduced )
Dormant bug
(discovered
many versions
after it was
introduced )
Bug 1
…
Zzz..Zzz..
Bug 3
5. Dormant bugs affect perceived
software quality
5
VersionA
Version
B
Version A has 5 bugs Version B has 10 bugs
6. Dormant bugs affect perceived
software quality
6
VersionA
Version
B
Version A has 5 bugs Version B has 10 bugs
Version B has 8
dormant bugs!
These dormant bugs were
introduced in version A
7. Case study on 20 systems
7
Studied 20 open-source systems in the Apache Foundation
Analyzed all versions
System LOCs: 61K ~ 1.5M
# dormant
bugs
29%
# non-
dormant
bugs
71%
8. Many bugs are actually dormant bugs
8
VersionA
Version
C
Bug introduced
in Version A
33% of bugs become
dormant bugs!
VersionA
Bugs reported in
Version A
Many post-release bugs are actually
introduced in MUCH EARLIER versions!
20% of bugs are
dormant bugs from
prior versions
9. RQ1: How quickly are
dormant bugs fixed?
Research questions
9
RQ2: What is the size
of a dormant bug fix?
RQ3: Who fixes
dormant bugs?
RQ4: What are the root
causes of dormant bug
fixes?
10. RQ1: How quickly are
dormant bugs fixed?
Research questions
10
RQ2: What is the size
of a dormant bug fix?
RQ3: Who fixes
dormant bugs?
RQ4: What are the root
causes of dormant bug
fixes?
11. Dormant bugs are fixed faster and the
fixes are larger
11
Dormant Non-dormant
Take 5 days to fix
(median)
Takes 60% more
time to fix
Dormant Non-dormant
Fix size is 19
LOC (median)
Fix size is 50%
smaller
12. RQ1: How quickly are
dormant bugs fixed?
Research questions
12
RQ2: What is the size
of a dormant bug fix?
RQ3: Who fixes
dormant bugs?
RQ4: What are the root
causes of dormant bug
fixes?
13. RQ1: How quickly are
dormant bugs fixed?
Research questions
13
RQ2: What is the size
of a dormant bug fix?
RQ3: Who fixes
dormant bugs?
RQ4: What are the root
causes of dormant bug
fixes?
14. Dormant fixers are more experienced
14
Dormant Non-dormant
#ofpriorcommitsofbugfixers
15. RQ1: How quickly are
dormant bugs fixed?
Research questions
15
RQ2: What is the size
of a dormant bug fix?
RQ3: Who fixes
dormant bugs?
RQ4: What are the root
causes of dormant bug
fixes?
16. RQ1: How quickly are
dormant bugs fixed?
Research questions
16
RQ2: What is the size
of a dormant bug fix?
RQ3: Who fixes
dormant bugs?
RQ4: What are the root
causes of dormant bug
fixes?
17. Manual study of the root causes
17
All bugs
Randomly sampled 357
dormant bugs and 357
non-dormant bugs
Manually classify the
bugs