DRONE is a tool that detects and repairs defects in API documentation by analyzing documentation directives and source code. It uses natural language processing techniques to extract constraints from documentation and builds abstract syntax trees from code to classify parameter usage. DRONE was evaluated on 8 Java libraries and Android APIs, detecting 1291 real defects with a precision over 76%, recall over 84%, and F-measure over 80%. It also provides repair recommendations for the detected defects based on predefined templates.

1. DRONE: A Tool to Detect and Repair
Directive Defects in Java APIs
Documentation
Sebastiano
Panichella
Xin
Yan
Taolue
Chen
Harald
Gall
Yu
Zhou

2. Outline
2
DRONE
Context:
Proposed Solution based on:
Case Study:
APIs Usage in OSS
and Industrial Projects
- NLP Approaches;
- Static Analysis Techniques;
- Logic reasoning;
Assessment of DRONE on
Documentation and Code
8 Java Libraries
Documentation Code

3. Open Source (OS)
and Industrial Projects
3
“Social networks like Facebook or Pinterest, or utilities like
Google Maps or Dropbox are popular examples of APIs providers.”

4. Open Source (OS)
and Industrial Projects
4
“Social networks like Facebook or Pinterest, or utilities like
Google Maps or Dropbox are popular examples of APIs providers.”

5. Open Source (OS)
and Industrial Projects
5“APIs are great time savers for Developers…”

6. 6
Difficult
to Understand
describe
Information
Needed
Ideally….
“API documents represent one of the most
important references for developers…”
Source Code
APIs Documents
Application Programming
Interface (API) Documents

7. 7
Software Changes over the Time
“…as consequence the original documentation tend to be
incomplete and inconsistent with the source code…”
Insufficient
Information
Source Code
Difficult
to Understand
APIs Documents
Inconsistent withComing back
to the reality...
Inconsistent/
Incomplete

8. 8
Source Code
APIs Documents
Inconsistent/
Incomplete
API Document Defect: Example 1
Class:
TextLayout.
Method:
getBlackBoxBounds(int firstEndpoint, int secondEndpoint)
API Document JDK-1.8

9. 9
Source Code
APIs Documents
Inconsistent/
Incomplete
API Document Defect: Example 1
Class:
TextLayout.
Method:
getBlackBoxBounds(int firstEndpoint, int secondEndpoint)
API Document JDK-1.8
https://docs.oracle.com/javase/8/docs/api/java/awt/font/TextLayout.html

10. 10
Source Code
APIs Documents
Inconsistent/
Incomplete
API Document Defect: Example 1
API Document JDK-1.8
Class:
TextLayout.
Method:
getBlackBoxBounds(int firstEndpoint, int secondEndpoint)
https://docs.oracle.com/javase/8/docs/api/java/awt/font/TextLayout.html

11. 11
Source Code
APIs Documents
Inconsistent/
Incomplete
API Document Defect: Example 1
API Document JDK-1.8
Class:
TextLayout.
Method:
getBlackBoxBounds(int firstEndpoint, int secondEndpoint)
https://docs.oracle.com/javase/8/docs/api/java/awt/font/TextLayout.html

12. 12
Source Code
APIs Documents
Inconsistent/
Incomplete
API Document Defect: Example 1
API Document JDK-1.8
Class:
TextLayout.
Method:
getBlackBoxBounds(int firstEndpoint, int secondEndpoint)
————————————-—————————————-
————————————-—————————————-
https://docs.oracle.com/javase/8/docs/api/java/awt/font/TextLayout.html

13. 13
Source Code
APIs Documents
Inconsistent/
Incomplete
API Document Defect: Example 1
API Document JDK-1.8
Class:
TextLayout.
Method:
getBlackBoxBounds(int firstEndpoint, int secondEndpoint)
————————————-—————————————-
————————————-—————————————-
https://docs.oracle.com/javase/8/docs/api/java/awt/font/TextLayout.html

14. 14
Source Code
APIs Documents
Incomplete
API Document Defect: Example 1
API Document JDK-1.8
Class:
TextLayout.
Method:
getBlackBoxBounds(int firstEndpoint, int secondEndpoint)
————————————-—————————————-
————————————-—————————————-
https://docs.oracle.com/javase/8/docs/api/java/awt/font/TextLayout.html

15. ————————————-—————————————-
————————————-—————————————-
15
Source Code
APIs Documents
Inconsistent
API Document Defect: Example 2
API Document JDK-1.8
Class:
InputEvent
Method:
getMaskForButton(int button)
https://docs.oracle.com/javase/8/docs/api/java/awt/event/InputEvent.html

16. 16
Source Code
APIs Documents
Inconsistent
API Document Defect: Example 2
API Document JDK-1.8
Class:
InputEvent
Method:
getMaskForButton(int button)
————————————-—————————————-
————————————-—————————————-
https://docs.oracle.com/javase/8/docs/api/java/awt/event/InputEvent.html

17. 17
Source Code
APIs Documents
Inconsistent
API Document Defects are Frequent
API Document JDK-1.8
Class:
InputEvent
Method:
getMaskForButton(int button)
————————————-—————————————-
————————————-—————————————-
https://docs.oracle.com/javase/8/docs/api/java/awt/event/InputEvent.html
“…and tend to be discovered and fixed after long time…”
http://stackoverflow.com/questions/2967303/inconsistency-in-java-util-concurrent-future

18. 18
DRONE
Detect and Repair of dOcumentatioN dEfects)
Code
API
Document
Software
Artifacts
AST Parsing
Pre-Process
and POS
Tagging
Defect
Reports
Control Flow-
Based Constraint
Analysis
SMT
Solver
Dependency
Parsing and
Pattern Analysis
Code Constraint
FOL Generating
Doc Constraint
FOL Generating
HeuristicsHeuristics

19. 19
DRONE
DetectoR of dOcumentatioN dEfects

20. 20
DRONE
DetectoR of dOcumentatioN dEfects
“…we consider 4 cases of parameter usage constraints”

21. 21
DRONE
DetectoR of dOcumentatioN dEfects
“…we consider 4 cases of parameter usage constraints”
1) “Nullness not allowed” 2) “Nullness allowed” NULL

22. 22
DRONE
DetectoR of dOcumentatioN dEfects
“…we consider 4 cases of parameter usage constraints”
1) “Nullness not allowed” 2) “Nullness allowed”
3) “Type restriction” 4) “Range limitation”
NULL
T

23. Previous Work
ICSE 2017 TSE 2018

24. 24
Step 1. Construct AST

25. 25
Step 1. Construct AST
Source Code

26. 26
Step 1. Construct AST
Source Code
c
c(i+1)
…….
…….

27. 27
Step 1. Construct AST
1) for each method “m” the 2) its code call graph G
c(i+1);
c
= { call(m,c) }
c(i+1)
…….
…….
“…and extracts”
27

28. 28
Step 1. Construct AST
28
1) for each method “m” the 2) its code call graph G
c(i+1);
c
= { call(m,c) }
c(i+1)
…….
“…and extracts”

29. 29
Step 1. Construct AST
29

30. 30
Step 1. Construct AST
30
c
For each

31. 31
Step 1. Construct AST
31
c
For each
we collect

32. 32
Step 1. Construct AST
32
c(i+1);c(i+1)
…….
…….
c
For each
we classify the
1) “Nullness not allowed”
2) “Nullness allowed”
3) “Type restriction”
NULL
T

33. 33
Step 1. Construct AST
33
c(i+1);c(i+1)
…….
…….
1) “Nullness not allowed”
2) “Nullness allowed”
3) “Type restriction”
NULL
T
4) “Range limitation”

34. 34
Extract Constraints from Directives in
API Documents
Natural Language
Parsing

35. 35
Extract Constraints from Directives in
API Documents
Natural Language
Parsing
“Recurrent Linguistic Patterns (LPs)…”
Di Sorbo et al. ASE 2015

36. 36
Example
“ @exception NullPointerException if the item’s value
is equal to < code> null< /code>”
“Recurrent Linguistic Patterns (LPs)…”
Di Sorbo et al. ASE 2015
Class: InputEvent java.awt.Choice Method: addItem(String item)

37. 37
Example
“ @exception NullPointerException if the item’s value
is equal to < code> null< /code>”
{@exception NullPointerException }
Class: InputEvent java.awt.Choice Method: addItem(String item)
“Linguistic Pattern (LP)…”
“item equal
to null”
Di Sorbo et al. ASE 2015

38. 38
Definition of NLP Heuristic
“Linguistic Pattern (LP)…”
“item equal
to null”
Steps:
1) Considering the relevant details;
2) Generalizing some information;
3) Ignoring useless information
NLP Heuristic:
Di Sorbo et al. ASE 2015
NLP Heuristic:
1) if the item’s value is equal to null”

39. 39
Definition of NLP Heuristic
{@exception NullPointerException }
“Linguistic Pattern (LP)…”
“item equal
to null”
Steps:
1) Considering the relevant details;
2) Generalizing some information;
3) Ignoring useless information
NLP Heuristic:
1) if the item’s value is equal to null”
Di Sorbo et al. ASE 2015

40. 40
Definition of NLP Heuristic
{@exception NullPointerException }
“Linguistic Pattern (LP)…”
“item equal
to null”
Steps:
1) Considering the relevant details;
2) Generalizing some information;
3) Ignoring useless information
NLP Heuristic:
1) if the item’s value is equal to null”
2) if the (subj)’s value is equal to null”
Di Sorbo et al. ASE 2015

41. 2) if the (subj)’s value is equal to null”
41
Definition of NLP Heuristic
{@exception NullPointerException }
“Linguistic Pattern (LP)…”
“item equal
to null”
Steps:
1) Considering the relevant details;
2) Generalizing some information;
3) Ignoring useless information
NLP Heuristic:
1) if the item’s value is equal to null”
3) if the (subj)’s value is equal to null”
Di Sorbo et al. ASE 2015

42. 2) if the (subj)’s value is equal to null”
42
Definition of NLP Heuristic
“Linguistic Pattern (LP)…”
“item equal
to null”
NLP Heuristic:
1) if the item’s value is equal to null”
3) if the (subj)’s value is equal to null”
64 NLP Heuristics
Steps:
1) Considering the relevant details;
2) Generalizing some information;
3) Ignoring useless information
Di Sorbo et al. ASE 2015

43. 3) if the (subj)’s value is equal to null”
2) if the (subj)’s value is equal to null”
43
NLP Heuristic:
1) if the item’s value is equal to null”
64 NLP Heuristics
3) if the (subj)’s value is equal to null”
Steps:
1) Considering the relevant details;
2) Generalizing some information;
3) Ignoring useless information
Generation of FOL Constraints

44. 3) if the (subj)’s value is equal to null”
2) if the (subj)’s value is equal to null”
44
Generation of FOL Constraints
NLP Heuristic:
1) if the item’s value is equal to null”
64 NLP Heuristics
3) if the (subj)’s value is equal to null”
(subj) = null
Steps:
1) Considering the relevant details;
2) Generalizing some information;
3) Ignoring useless information

45. 45
?
SMT Solver

46. 46
DRONE
Demo
DRONE UI—defect detection view !

47. 47
Defects Repair:!
Based on the defect report generated from the previous part,
DRONE gives repair recommendations based on the predefined
templates.!
Repair Recommendation
Templates!
DRONE
Demo

48. 48
DRONE
DRONE UI—defect repair view !

49. 49
Case Studies
Documentation Code

50. 50
Two Case Studies
Case Study 1
8 JDK Libraries
Case Study 2
Android APIs

51. 51
Case Study 1
8 JDK Libraries
Android APIs

52. 52
8 JDK Libraries
Results of Case Study 1: JDK APIs
Case Study 1
DRONE
?
1540 Potential Defects
Android APIs

53. 53
8 JDK Libraries
Results of Case Study 1: JDK APIs
Case Study 1
1291 Real Defects
DRONE
?
1540 Potential Defects
Android APIs

54. 54
8 JDK Libraries
Results of Case Study 1: JDK APIs
Case Study 1
1291 Real Defects
DRONE
?
1540 Potential Defects
Precision > 0.76
Recall > 0.84
F-measure > 0.80
Android APIs

55. 55
Case Study 2
Questions Designed to Evaluate
Repair Recommendation

56. 56
Case Study 2
Result Distribution
Questions Designed to Evaluate
Repair Recommendation

57. Conclusion & Future Work
“API documents represent one of the most
important references for developers…”

58. Conclusion & Future Work
Documentation Code
“API documents represent one of the most
important references for developers…”
“Analyzing APIs Documentation and Code to Detect Directive Defects”. ICSE 2017
DRONE

59. Conclusion & Future Work
Documentation Code
With DRONE we analyzed
8 java libraries and The Android APIs
detecting around 1291 of
API documentation defects, providing
repair solutions for them.
with high precision, recall, and F-
measure
“API documents represent one of the most
important references for developers…”
DRONE