Poster: Improving Bug Localization with Report Quality Dynamics and Query Reformulation

1. IMPROVING BUG LOCALIZATION WITH REPORT QUALITY
DYNAMICS AND QUERY REFORMULATION
{MD. MASUDUR RAHMAN AND CHANCHAL K. ROY} UNIVERSITY OF SASKATCHEWAN, COMPUTER SCIENCE
ABSTRACT
In this poster paper, we present a large empirical
study using 5,500 bug reports from eight systems
and replicating three existing studies. Our find-
ings empirically demonstrate how quality dy-
namics of bug reports affect the performances of
the contemporary IR-based bug localizations. Ex-
isting techniques do not perform well if (1) a bug
report lacks rich structured information such as
relevant program entity names, and (2) the bug
report contains excessive structured information
such as stack traces. Our preliminary findings
also suggest that context-aware query reformulations
might help overcome such limitations.
TERM WEIGHT CALCULATION
TF −IDF(t) =
∀d∈DRF
(1+log(ft,d))×log
|D|
nt
Vi = {Ci, Mi},
Ei = {Ci ↔ Mi} ∪ {Ci → Cj, Mi → Mj} | j = i − 1
V =
N
i=1
{Vi}, E =
N
i=1
{Ei}, GST = (V, E)
S(Vi) = (1−ψ)+ψ
j In(Vi)
S(Vj)
|Out(Vj)|
(0 ≤ ψ ≤ 1)
SCHEMATIC DIAGRAM OF THE EMPIRICAL STUDY
Bug report
collection
Bug report
clustering
Clustered
bug reports
Project
codebase
BLUiR
BugLocator
LOBSTER
Result analysis
+ answering RQs
Findings
and insights
CONTACT INFORMATION
Web www.usask.ca/∼masud.rahman
Email masud.rahman@usask.ca
Twitter @masud2336
Phone +1 (306) 241 9293
CONCLUSION & FUTURE RESEARCH
• A large-scale empirical study pointing out
that state-of-the-art IR-based techniques are
not robust to various types of bug reports.
• Quality of the bug report is a major factor.
• Appropriate reformulation of the report con-
tents is warranted prior to bug localization.
• Future research can develop techniques that
take bug report quality into consideration.
REFERENCES
[1] M. M. Rahman and C. K. Roy. Poster: Improv-
ing bug localization with report quality dynamics
and query reformulation. In Proc. ICSE-C, page 02,
Gothenburg, Sweden, May 2018.
RESEARCH QUESTIONS & ANSWERS
• RQ1: How do existing IR-based bug local-
ization techniques perform with the bug re-
ports containing excessive amount of struc-
tured information (e.g., stack traces)?
• RQ2: How do existing IR-based techniques
perform with the bug reports containing
neither program element names nor stack
traces (i.e., only unstructured regular texts)?
• RQ3: Does a single technique perform si-
multaneously well with the bug reports
from both groups?
Figure 1: MAP@K of (a) Baseline (Lucene), (b) BugLocator, (c) BLUiR, and (d) LOBSTER with bug reports containing
excessive structured information (e.g., stack traces)
Figure 2: Hit@10 of (a) Baseline (Lucene), (b) BugLoca-
tor, (c) BLUiR, and (d) LOBSTER with bug reports con-
taining only regular texts
Figure 3: MAP@10 of all four techniques with bug re-
ports containing (a) stack traces, (b) natural language
texts only, (c) program elements, and (d) all bug reports
TRACE GRAPH DEVELOPMENT
Table 1: A Noisy Bug Report
Title: should be able to cast “null"
Bug ID: 31637, Project: eclipse.jdt.debug
Description: When trying to debug an application the variables
tab is empty. Also when I try to inspect or display a variable,
I get following error logged in the eclipse log file:
java.lang.NullPointerException
at org.eclipse.jdt.internal.debug.core.
model.JDIValue.toString(JDIValue.java:362)
at org.eclipse.jdt.internal.debug.eval.ast.
instructions.Cast.execute(Cast.java:88)
at org.eclipse.jdt.internal.debug.eval.ast.engine.
Interpreter.execute(Interpreter.java:44)
at org.eclipse.jdt.internal.debug.eval.ast.engine.
........................................ (8 more).......................................
Cast access
InterpreterJDIValue
toString run
runEvaluation
doEvaluation
EvaluationThread
execute
JDIThread
Thread
EvaluationThread
toString
JDIValue
run
execute
Figure 4: Trace graph of stack traces in Table 1
QUERY REFORMULATION
Table 2: An Example of Query Reformulation
Technique Group Query Terms QE
Baseline
BRST
127 terms from Table 1 after
preprocessing, Bug ID# 31637,
eclipse.jdt.debug
53
Proposed NullPointerException + “Bug
should be able to cast null" +
{JDIValue toString execute
EvaluationThread run}
01
Baseline
BRP E
195 terms (after preprocessing) from
Bug ID# 15036, eclipse.jdt.core
27
Proposed {astvisitor post postvisit previsit pre
file post pre astnode visitor}
01
Baseline
BRNL
32 terms after preprocessing, Bug ID#
475855, eclipse.jdt.ui
30
Proposed Preprocessed report texts
+ {compliance create
preference add configuration
field dialog annotation}
01
Table 3: A Poor Bug Report
Title: [preferences] Mark Occurences Pref Page
Bug ID: 187316, Project: eclipse.jdt.ui
Description: There should be a link to the pref page
on which you can change the color. Namely: Gener-
al/Editors/Text Editors/Annotations. It’s a pain in
the a** to find the pref if you do not know Eclipse’s
preference structure well.