This document introduces MUBench, a benchmark for evaluating API misuse detectors. MUBench contains 89 real-world examples of API misuse collected from open source projects on GitHub and SourceForge, as well as from a survey. Each misuse example includes a description, the types involved, and metadata linking to the project and fixing commit. The benchmark is hosted on GitHub and aims to help researchers test and compare different approaches for detecting API misuses.

1. A Benchmark for API-Misuse Detectors
Sven Amann•
, Sarah Nadi•
, Hoan A. Nguyen◦
, Tien N. Nguyen◦
, and Mira Mezini•
•
Technische Universität Darmstadt ◦
Iowa State University

2. List<String> l = …
for (String s : l) {
if (s.startsWith(“.”)) {
l.remove(s);
}
}
API Misuse
2

3. API-Misuse Detectors
3
2.
Find Rare Violations
(Anomalies)
1.
Obtain Patterns

4. 4

5. 5
Bug Datasets
Mining
GitHub & SourceForge Survey

6. 6
Project
Bug Report
Description
Involved Types
Characteristics
Repo + Fixing Commit
<misuse>.yml

7. 7
Contains 89 Misuses
68.5% Crashes
github.com/stg-tud/MUBench/tree/MSR16
From 33 Projects
and a Survey

8. 8
git
svn
synt
working
copy
version
with misuse

9. A Benchmark for
API-Misuse Detectors
github.com/stg-tud/MUBench
Sven Amann
amann@cs.tu-darmstadt.de