1. Bench4BL: Reproducibility Study on
the Performance of IR-Based Bug
Localization
Jaekwon Lee1, Dongsun Kim1, Tegawendé F. Bissyandé1,
Woosung Jung2, Yves Le Traon1
1SnT, University of Luxembourg - Luxembourg
2Seoul National University of Education - South Korea

2. Bug Localization
!2

3. Bug Localization
!3
Where should we fix?

4. Bug Localization
!4
Model
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
Bug Report
………..
…. …..
…..….
……..
….
..
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
A set of code files
Bug Localization

5. F(x)
Test Case
Test Case
Test Case
01: MultiMap<Method, Long> duration = new MultiMap<Method, Long>();
02: // run all benchmarks in same order, recording duration
03: for (Method m : benchmarks) {
04: System.err.println("# "+m.getName()+" benchmarking");
05: List<Integer> reps = getReps(min_reps, m);
06: for (int r : reps) {
07: System.gc();
08: long start = System.nanoTime();
09: m.invoke(suite,r);
10: long stop = System.nanoTime();
11: duration.map(m, stop - start);
12: }
13: }
Function
01: MultiMap<Method, Long> duration = new MultiMap<Method, Long>();
02: // run all benchmarks in same order, recording duration
03: for (Method m : benchmarks) {
04: System.err.println("# "+m.getName()+" benchmarking");
05: List<Integer> reps = getReps(min_reps, m);
06: for (int r : reps) {
07: System.gc();
08: long start = System.nanoTime();
09: m.invoke(suite,r);
10: long stop = System.nanoTime();
11: duration.map(m, stop - start);
12: }
13: }
Function
Fault Localization
Bug Localization
!5
Bug Localization
Model
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
Bug Report
………..
…. …..
…..….
……..
….
..
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
A set of code files

6. Bug Localization
F(x)
Test Case
Test Case
Test Case
01: MultiMap<Method, Long> duration = new MultiMap<Method, Long>();
02: // run all benchmarks in same order, recording duration
03: for (Method m : benchmarks) {
04: System.err.println("# "+m.getName()+" benchmarking");
05: List<Integer> reps = getReps(min_reps, m);
06: for (int r : reps) {
07: System.gc();
08: long start = System.nanoTime();
09: m.invoke(suite,r);
10: long stop = System.nanoTime();
11: duration.map(m, stop - start);
12: }
13: }
Function
01: MultiMap<Method, Long> duration = new MultiMap<Method, Long>();
02: // run all benchmarks in same order, recording duration
03: for (Method m : benchmarks) {
04: System.err.println("# "+m.getName()+" benchmarking");
05: List<Integer> reps = getReps(min_reps, m);
06: for (int r : reps) {
07: System.gc();
08: long start = System.nanoTime();
09: m.invoke(suite,r);
10: long stop = System.nanoTime();
11: duration.map(m, stop - start);
12: }
13: }
Function
Fault Localization
Bug Localization
!6
Model
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
Bug Report
………..
…. …..
…..….
……..
….
..
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
…..
…..…..
…..…..
…..…..
….…..Java
A set of code files

7. …..…..
…..…..
…..…..
…..…..
…..…..
Source
Codes
……..……..
……. ……..
…..……..
……..……..
…….. …….
Bug
Report
Information Retrieval based
Bug Localization (IRBL)
!7

8. …..…..
…..…..
…..…..
…..…..
…..…..
Source
Codes
……..……..
……. ……..
…..……..
……..……..
…….. …….
Bug
Report
NL tokens
Code elements
Meta Info.
NL tokens
Code elements
Meta Info.
Extracting Features
Extracting Features
Information Retrieval based
Bug Localization (IRBL)
!8

9. Feature
Vector
…..…..
…..…..
…..…..
…..…..
…..…..
Source
Codes
……..……..
……. ……..
…..……..
……..……..
…….. …….
Bug
Report
….
Feature
Vectors
NL tokens
Code elements
Meta Info.
NL tokens
Code elements
Meta Info.
Extracting Features
Extracting Features
Information Retrieval based
Bug Localization (IRBL)
!9

10. Feature
Vector
…..…..
…..…..
…..…..
…..…..
…..…..
Source
Codes
……..……..
……. ……..
…..……..
……..……..
…….. …….
Bug
Report
Recommend
Code Files
…..
…..…..
…..…..
…..…..
…..
…..…..
…..…..
…..…..
…..
…..…..
…..…..
…..…..
…..
…..…..
…..…..
…..…..
1
2
3
N
….
….
Feature
Vectors
NL tokens
Code elements
Meta Info.
NL tokens
Code elements
Meta Info.
Extracting Features
Extracting Features
Comparing Similarity
& Ranking
Information Retrieval based
Bug Localization (IRBL)
!10

11. !11
Is there any issue?

12. !12
Are these results mature enough?
Not enough maturity of performance
Subjects BRTracer BLUiR AmaLgam Locus
ZXing 0.445 0.380 0.410 0.502
SWT 0.467 0.560 0578 0.640
AspectJ 0.264 0.263 0.271 0.320
PDE 0.367 0.349 0.322 0.422
JDT 0.232 0.277 0.282 0.359
(metric : MAP)

13. Are the subjects still usable?
!13
PDE
Eclipse
ZXing
AspectJ
JDT
SWT
98
286
20
#Reports Period
2004 - 2016
2004 - 2010
2002 - 2006
2010 - 2010
Subject
Out-of-dated subjects
60
98

14. Are the subjects still usable?
!14
PDE
Eclipse
ZXing
AspectJ
JDT
SWT
98
286
20
#Reports Period
2004 - 2016
2004 - 2010
2002 - 2006
2010 - 2010
Subject
Out-of-dated subjects
60
98

15. Evaluation Configuration?
!15
Inconsistent evaluation settings
BugLocator
BLIA
Locus
AmaLgam
BRTracer
BLUiR
Version Matching
Test file inclusion

16. Study Design
!16

17. Experiment
Data Set
RQ1: To what extent do IRBL techniques
perform on up-to-date subjects?
Research Questions
!17

18. Experiment
Data Set
Experiment
Configuration
RQ1: To what extent do IRBL techniques
perform on up-to-date subject?
RQ2: What is the impact of version
matching on the performance of IRBL
techniques?
RQ3: To what extent are IRBL techniques
sensitive to the inclusion of test code files?
Research Questions
!18

19. Experiment
Data Set
Experiment
Configuration
Potential
Improvement
RQ1: To what extent do IRBL techniques
perform on up-to-date subject?
RQ2: What is the impact of version
matching on the performance of IRBL
techniques?
RQ3: To what extent are IRBL techniques
sensitive to the inclusion of test code files?
RQ4: What potential performance gain can
be reached by leveraging duplicate bug
reports?
Research Questions
!19

20. BugLocator
(ICSE 2012)
BLIA
(APSEC 2015)
Locus
(ICSE 2016)
AmaLgam
(ICPC 2014)
BRTracer
(ICSME 2014)
BLUiR
(ASE 2013)
IRBL Features Sub Modules
Bug report fixing historyFull text
Code
segmentations
Identifiers
Identifiers
Identifiers
Identifiers
Bug report fixing history
Bug report fixing history, Revision history
Revision history
Bug report fixing history
Stack Trace Analysis, Revision history
Bug report fixing history,
Stack Trace Analysis
IRBL Techniques we used
!20

21. Subjects
!21
20+
Written in Java
Publicly available
bug reports
20 source code files
in one of its version

22. Subjects
46
Projects
New Subjects
9,459
Bug
Reports
………..
…. …..
…..….
……..
….
..
5
Projects
558
Bug
Reports
………..
…. …..
…..….
……..
….
..
Old Subjects
!22

23. Subjects
46
Projects
New Subjects
690
Major
Versions
9,459
Bug
Reports
………..
…. …..
…..….
……..
….
..
………..
…. …..
…..….
……..
….
..
………..
…. …..
…..….
……..
….
..
807
Duplicate
Reports
5
Projects
5
Major
Versions
558
Bug
Reports
………..
…. …..
…..….
……..
….
..
………..
…. …..
…..….
……..
….
..
………..
…. …..
…..….
……..
….
..
136
Duplicate
Reports
Old Subjects
!23

24. !24
………..
…. …..
…..….
……..
….
..
…..…..
…..…..
…..…..
…..….…..
Java
…..…..
…..…..
…..…..
…..….…..
Java
…..…..
…..…..
…..…..
…..….…..
Java
………..
…. …..
…..….
……..
….
..
………..
…. …..
…..….
……..
….
..
Bug Oracle
New Subjects Old Subjects
………..
…. …..
…..….
……..
….
..
…..…..
…..…..
…..…..
…..….…..
Java
…..…..
…..…..
…..…..
…..….…..
Java
…..…..
…..…..
…..…..
…..….…..
Java
………..
…. …..
…..….
……..
….
..
………..
…. …..
…..….
……..
….
..
Bug Oracle
VS.
Single version matching
Test file included
Configuration
RQ1:
The use of old vs. new subjects

25. Single version
Matching
Multiple version
Matching
!25
VS.
Configuration
New subjects
Test files included
RQ2:
The importance of version matching

26. !26
………..
…. …..
…..….
……..
….
..
…..…..
…..…..
…..…..
…..….…..
Java
…..…..
…..…..
…..…..
…..….…..
Java
…..…..
…..…..
…..…..
…..….…..
Java
………..
…. …..
…..….
……..
….
..
………..
…. …..
…..….
……..
….
..
Bug Oracle
Test File Included Test File Excluded
VS.
+Test
………..
…. …..
…..….
……..
….
..
…..…..
…..…..
…..…..
…..….…..
Java
…..…..
…..…..
…..…..
…..….…..
Java
…..…..
…..…..
…..…..
…..….…..
Java
………..
…. …..
…..….
……..
….
..
………..
…. …..
…..….
……..
….
..
Bug Oracle
CAMEL-12558: Transacted and Policy should not have outputs
M main/java/org/apache/camel/model/PolicyDefinition.java
M main/java/org/apache/camel/model/TransactedDefinition.java
A test/java/org/apache/camel/catalog/CamelCatalog.java
A main/java/org/apache/camel/tools/apt/CoreEipAnnotationPrint.java
Added camel-web3j Spring-boot test
A test/java/org/apache/camel/itest/springboot/CamelWeb3jTest.java
Update GoogleBigQueryProducer.java
M main/java/org/apache/camel/component/GoogleBigQueryProducer.java
Configuration
Multiple version matching
New subjects
Commit Log
RQ3:
The impact of test file inclusion

27. !27
Master reports Merged reportsDuplicate reports
………..
…. …..
…..….
……..
….
..
…..…..
…..…..
…..…..
…..….…..
Java
…..…..
…..…..
…..…..
…..….…..
Java
…..…..
…..…..
…..…..
…..….…..
Java
………..
…. …..
…..….
……..
….
..
………..
…. …..
…..….
……..
….
..
Bug Oracle
(Master reports)
………..
…. …..
…..….
……..
….
..
…..…..
…..…..
…..…..
…..….…..
Java
…..…..
…..…..
…..…..
…..….…..
Java
…..…..
…..…..
…..…..
…..….…..
Java
………..
…. …..
…..….
……..
….
..
………..
…. …..
…..….
……..
….
..
Bug Oracle
(Duplicate reports)
Bug Oracle
(Merged reports)
…..…..
…..…..
…..…..
…..….…..
Java
…..…..
…..…..
…..…..
…..….…..
Java
…..…..
…..…..
…..…..
…..….…..
Java
………..
…. …..
…..….
……..
….
..
…..….
……..
….
………..
…. …..
…..….
……..
….
..
…..….
……..
….
………..
…. …..
…..….
……..
….
..
…..….
……..
….
Configuration
For the all subjects
Including test files in Bug Oracle
RQ4:
Leveraging duplicate bugs reports

28. Experiment Results
!28

29. Metrics
MAP
MRR
MAP =
1
M
MX
j=1
AP(j)
MRR =
1
M
MX
i=1
1
f-ranki
MAP =
1
M
MX
j=1
AP(j)
MRR =
1
M
MX
i=1
1
f-ranki
!29
Mean Average Precision
Mean Reciprocal Rank

30. ●●
●●
0.359
0.35
0.359
0.365
0.363
0.38Locus
BLIA
AmaLgam
BLUiR
BRTracer
BugLocator
0.00 0.25 0.50 0.75 1.00
Distribution of MAP values of all subjects for each techniques
0.455
0.501
0.43
0.516
0.497
0.506Locus
BLIA
AmaLgam
BLUiR
BRTracer
BugLocator
0.00 0.25 0.50 0.75 1.00
Distribution of MRR values of all subjects for each techniques
Baseline Performance
!30

31. ●●
●●
0.359
0.35
0.359
0.365
0.363
0.38Locus
BLIA
AmaLgam
BLUiR
BRTracer
BugLocator
0.00 0.25 0.50 0.75 1.00
Distribution of MAP values of all subjects for each techniques
0.455
0.501
0.43
0.516
0.497
0.506Locus
BLIA
AmaLgam
BLUiR
BRTracer
BugLocator
0.00 0.25 0.50 0.75 1.00
Distribution of MRR values of all subjects for each techniques
Baseline Performance
Bug localization still has much room for improvement.
!31

32. Technique
Old Subjects New Subjects
MAP MRR MAP MRR
BugLocator 0.2692 0.3985 ↗0.3052 ↗0.4223
BRTracer 0.2645 0.3664 ↗0.3330 ↗0.4690
BLUiR 0.3102 0.4556 0.2881 0.3869
AmaLgam 0.2950 0.4072 0.2906 0.3899
BLIA 0.2935 0.4242 ↗0.3014 0.4155
Locus 0.2641 0.3399 ↗0.3289 ↗0.4430
Single version matching
Test files included
Summary of MAP/MRR of IRBL techniques
!32
Configuration
RQ1:
The use of old vs. new subjects

33. Summary of MAP/MRR of IRBL techniques
Not over-fitted to old subjects
Technique
Old Subjects New Subjects
MAP MRR MAP MRR
BugLocator 0.2692 0.3985 ↗0.3052 ↗0.4223
BRTracer 0.2645 0.3664 ↗0.3330 ↗0.4690
BLUiR 0.3102 0.4556 0.2881 0.3869
AmaLgam 0.2950 0.4072 0.2906 0.3899
BLIA 0.2935 0.4242 ↗0.3014 0.4155
Locus 0.2641 0.3399 ↗0.3289 ↗0.4430
!33
Single version matching
Test files included
Configuration
RQ1:
The use of old vs. new subjects

34. !34
Summary of MAP/MRR of IRBL techniques
Technique
Single Version Multiple Version
MAP MRR MAP MRR
BugLocator 0.3052 0.4223 ↗0.3713 ↗0.5075
BRTracer 0.3330 0.4690 ↗0.3992 ↗0.5526
BLUiR 0.2881 0.3869 ↗0.3623 ↗0.4802
AmaLgam 0.2906 0.3899 ↗0.3657 ↗0.4840
BLIA 0.3014 0.4155 ↗0.3777 ↗0.5124
Locus 0.3289 0.4430 ↗0.4217 ↗0.5514
New subjects
Test files included
Configuration
RQ2:
The importance of version matching

35. New subjects
Test files included
Summary of MAP/MRR of IRBL techniques
The evaluation/execution of IRBL techniques should apply
multiple version matching
!35
Technique
Single Version Multiple Version
MAP MRR MAP MRR
BugLocator 0.3052 0.4223 ↗0.3713 ↗0.5075
BRTracer 0.3330 0.4690 ↗0.3992 ↗0.5526
BLUiR 0.2881 0.3869 ↗0.3623 ↗0.4802
AmaLgam 0.2906 0.3899 ↗0.3657 ↗0.4840
BLIA 0.3014 0.4155 ↗0.3777 ↗0.5124
Locus 0.3289 0.4430 ↗0.4217 ↗0.5514
Configuration
RQ2:
The importance of version matching

36. ConfigurationSummary of MAP/MRR of IRBL techniques
!36
Multiple version matching
New subjects
RQ3:
The impact of test file inclusion
Technique
Test files excluded Test files included
MAP MRR MAP MRR
BugLocator 0.3811 0.4647 0.3713 ↗0.5075
BRTracer 0.4141 0.5090 0.3992 ↗0.5526
BLUiR 0.3603 0.4385 ↗0.3623 ↗0.4802
AmaLgam 0.3633 0.4420 0.3657 ↗0.4840
BLIA 0.3902 0.4728 ↗0.3777 ↗0.5124
Locus 0.4146 0.5002 ↗0.4217 ↗0.5514

37. Summary of MAP/MRR of IRBL techniques
Technique
Test files excluded Test files included
MAP MRR MAP MRR
BugLocator 0.3811 0.4647 0.3713 ↗0.5075
BRTracer 0.4141 0.5090 0.3992 ↗0.5526
BLUiR 0.3603 0.4385 ↗0.3623 ↗0.4802
AmaLgam 0.3633 0.4420 0.3657 ↗0.4840
BLIA 0.3902 0.4728 ↗0.3777 ↗0.5124
Locus 0.4146 0.5002 ↗0.4217 ↗0.5514
!37
Including test files does not bring bias or noise
Configuration
Multiple version matching
New subjects
RQ3:
The impact of test file inclusion

38. RQ4:
Leveraging duplicate bugs reports
!38
Technique
Master Duplicate Merged
(Master+Duplicate)
MAP MRR MAP MRR MAP MRR
BugLocator 0.3503 0.5051 0.3259 0.4667 0.3502 ↗0.5249
BRTracer 0.3852 0.5508 0.3776 0.5430 0.3787 ↗0.5692
BLUiR 0.3159 0.4540 0.2804 0.4192 ↗0.3325 ↗0.4728
AmaLgam 0.3202 0.4581 0.2829 0.4223 ↗0.3327 ↗0.4725
BLIA 0.3518 0.4915 0.3231 0.4537 ↗0.3577 ↗0.5041
Locus 0.2915 0.4707 0.2871 ↗0.4724 ↗0.3042 ↗0.5021
Summary of MAP/MRR of IRBL techniques

39. !39
Summary of MAP/MRR of IRBL techniques
RQ4:
Leveraging duplicate bugs reports
Technique
Master Duplicate Merged
(Master+Duplicate)
MAP MRR MAP MRR MAP MRR
BugLocator 0.3503 0.5051 0.3259 0.4667 0.3502 ↗0.5249
BRTracer 0.3852 0.5508 0.3776 0.5430 0.3787 ↗0.5692
BLUiR 0.3159 0.4540 0.2804 0.4192 ↗0.3325 ↗0.4728
AmaLgam 0.3202 0.4581 0.2829 0.4223 ↗0.3327 ↗0.4725
BLIA 0.3518 0.4915 0.3231 0.4537 ↗0.3577 ↗0.5041
Locus 0.2915 0.4707 0.2871 ↗0.4724 ↗0.3042 ↗0.5021
Duplicate reports are complement master bug reports and
guarantee a minimum level of performance

40. Summary
!40

41. !41
Dataset Available
https://github.com/exatoa/Bench4BL

42. Bug Linking
!42

43. Bug-Code Linking
Bug Report
CAMEL-12558: Transacted and Policy should not have outputs
M main/java/org/apache/camel/model/PolicyDefinition.java
M main/java/org/apache/camel/model/TransactedDefinition.java
A test/java/org/apache/camel/catalog/CamelCatalog.java
A main/java/org/apache/camel/tools/apt/CoreEipAnnotationPrint.java
Added camel-web3j Spring-boot test
A test/java/org/apache/camel/itest/springboot/CamelWeb3jTest.java
Update GoogleBigQueryProducer.java
M main/java/org/apache/camel/component/GoogleBigQueryProducer.java
Code
Repository
Commit Log
!43

44. Bug-Code Linking
Bug Report
CAMEL-12558: Transacted and Policy should not have outputs
M main/java/org/apache/camel/model/PolicyDefinition.java
M main/java/org/apache/camel/model/TransactedDefinition.java
A test/java/org/apache/camel/catalog/CamelCatalog.java
A main/java/org/apache/camel/tools/apt/CoreEipAnnotationPrint.java
Added camel-web3j Spring-boot test
A test/java/org/apache/camel/itest/springboot/CamelWeb3jTest.java
Update GoogleBigQueryProducer.java
M main/java/org/apache/camel/component/GoogleBigQueryProducer.java
Code
Repository
Commit Log
!44

45. Bug Oracle
………..
…. …..
…..….
……..
….
..
main/java/org/apache/camel/model/PolicyDefinition.java
main/java/org/apache/camel/model/TransactedDefinition.java
test/java/org/apache/camel/catalog/CamelCatalog.java
main/java/org/apache/camel/tools/apt/CoreEipAnnotationPrint.java
………..
…. …..
…..….
……..
….
..
main/java/org/apache/camel/GoogleBigQueryProducer.java
………..
…. …..
…..….
……..
….
..
main/java/org/apache/camel/component/StringConcatenator.java
Bug Report 1
Bug Report 2
Bug Report 3
…….
!45

46. Version Matching
!46

47. Version Matching Strategy
Single version
Matching
!47
Previous Techniques

48. Version Matching Strategy
Single version
Matching
Multiple version
Matching
!48
Previous Techniques

49. Version Matching Approach
!49

50. Version Matching Approach
Selecting earliest version
!50

51. Test Case Inclusion
!51

52. Test File Inclusion
CAMEL-12558: Transacted and Policy should not have outputs
M main/java/org/apache/camel/model/PolicyDefinition.java
M main/java/org/apache/camel/model/TransactedDefinition.java
A test/java/org/apache/camel/catalog/CamelCatalog.java
A main/java/org/apache/camel/tools/apt/CoreEipAnnotationPrint.java
Added camel-web3j Spring-boot test
A test/java/org/apache/camel/itest/springboot/CamelWeb3jTest.java
Update GoogleBigQueryProducer.java
M main/java/org/apache/camel/component/GoogleBigQueryProducer.java
Code
Repository
Commit LogBugLocator
BLIA
Locus
AmaLgam
BRTracer
BLUiR
!52

53. Test File Inclusion
CAMEL-12558: Transacted and Policy should not have outputs
M main/java/org/apache/camel/model/PolicyDefinition.java
M main/java/org/apache/camel/model/TransactedDefinition.java
A test/java/org/apache/camel/catalog/CamelCatalog.java
A main/java/org/apache/camel/tools/apt/CoreEipAnnotationPrint.java
Added camel-web3j Spring-boot test
A test/java/org/apache/camel/itest/springboot/CamelWeb3jTest.java
Update GoogleBigQueryProducer.java
M main/java/org/apache/camel/component/GoogleBigQueryProducer.java
Code
Repository
Commit LogBugLocator
BLIA
Locus
AmaLgam
BRTracer
BLUiR
We remove
including “test” or “Test” in a path or filename
!53

54. Duplicate Report
!54

55. Duplicate Bug Reports
46
Projects
New Subjects
690
Major
Versions
9,459
Bug
Reports
………..
…. …..
…..….
……..
….
..
………..
…. …..
…..….
……..
….
..
………..
…. …..
…..….
……..
….
..
807
Duplicate
Reports
5
Projects
5
Major
Versions
558
Bug
Reports
………..
…. …..
…..….
……..
….
..
………..
…. …..
…..….
……..
….
..
………..
…. …..
…..….
……..
….
..
136
Duplicate
Reports
Old Subjects
!55

56. Duplicate Bug Reports
!56

57. Duplicate Bug Reports
MATH-760 MATH-1192 MATH-2022
MATH-760 MATH-1192
MATH-760 MATH-2022
!57