Learning to Spot and Refactor Inconsistent Method Names

logotype of the University
of Luxembourg
1 Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg
2 Department of Software Engineering, Chonbuk National University, South Korea
Kui Liu1, Dongsun Kim1, Tegawendé F. Bissyandé1, Taeyoung Kim2,
Kisub Kim1, Anil Koyuncu1, Suntae Kim2, Yves Le Traon1
Learning to Spot and Refactor
Inconsistent Method Names
29th May 2019

2
Programming with Libraries
LibraryA.java

2
LibraryA.java
getItem()
setObject(…)
…()

2
LibraryA.java
getItem()
setObject(…)
…()
Developers often
do not check the inside
of the method.

2
LibraryA.java
getItem()
setObject(…)
…()
Using a method relies on
its name (+ API document).
Developers often
do not check the inside
of the method.

3
A Method can Disguise
getPokemon( … )

3
getPokemon( … )
What I expect

3
getPokemon( … )
getPokemonRealMonster( … )
What I expect

3
getPokemon( … )
getPokemonRealMonster( … )
What I expect
What I actually get

4
Making the name consistent is NOT easy
https://www.itworld.com/article/2833265/
don-t-go-into-programming-if-you-don-t-
have-a-good-thesaurus.html
Naming Things
49%

5
Consequence of inconsistent names
There are 5K+ questions on naming issues in
stackoverflow.com.

6
Naming bugs are common
We found 183K+ commits addressing naming issues from
GitHub.com by a quick search with simple queries such as
“inconsistent, consistency, misleading, …”

7
Our Goals
Detect inconsistent
method names.
Repair the names to be
consistent.

8
Idea
Similar implementations would have similar names.

9
Idea
Similar implementations would have similar names.

10
Idea
Similar implementations, but different names.

How to ﬁnd similar
names/implementations?
12
Sim( , ) = ?

P P
17
Program Vectors
Program
Encoder

P P
17
Program Vectors
Program
Encoder
M1
M2
M3
M4

P P
17
Program Vectors
Program
Encoder
M1
M2
M3
M4
<9, 2, 3, …>
<7, 1, 6, …>
<2, 8, 3, …>
<0, 1, 8, …>

18
Method = Name + Body
getID(…)
{
for(…)
{
for(…)
{
for(…)
…
Similar
Names
Similar
Bodies
N
B

19
Method Name Embedding
findField
findMatchesHelper
containsTarget
containsField
findInstruction1
find, Field
find, Matches, Helper
contains, Target
contains, Field
find, Instruction1
Tokenized Names
(camel case, underscore)
Method
Names
Embedded
Vectors
Sentence2vec
(PV-DM)

20
return (String[]) list.toArray(new String[0]);
Method Body Embedding
Preprocessing: Program Serialization
Method Body:
[ReturnStatement, return,ArrayType, String[],Variable, listVar, Method, toArray,
ArrayCreation, new,ArrayType, String[], NumberLiteral,“0”]
Serialized AST:

21
Method Body:
Serialized AST:
...

22
Method Body:
Serialized AST:

23
Vectorization
Serialized AST:

24
Vectorization
Serialized AST:

24
Vectorization
Serialized AST:
Token Embedding
(Word2Vec)

24
Vectorization
Serialized AST:
Token Embedding
(Word2Vec)
<2, 6, 9, …> <8, 4, 1, …> <9, 0, 7, …> <2, 3, 0, …> … <7, 1, 2, …> …

25
Encoding (CNN-based)
icle has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSE.2018.288495
Transactions on Software Engineering
7
…
…
n ! k (a two-dimensional
numeric vector)
Input layer
C1: 4 feature maps
S1: 4 feature maps
C2: 6 feature maps
S2: 6 feature maps
Convolutional layer
Convolutional
layerSubsampling layer
Subsampling
layer Fully connected layers
Output
layer
Dense layer
Output is extracted
features
ReturnStatement
return
ArrayType
String[]
Variable
listVar
Method
toArray
ArrayCreation
new
ArrayType
String[]
NumberLiteral
“0”
0 0 0 0 0 00
0 0 0 0 0 00
Fig. 8: CNN architecture for extracting clustering features. C1 is the ﬁrst convolutional layer, and C2 is the second one. S1 is
2 6 9 ...
...
...
...
...
...
...
...
...
...
...
...
...
...
8 4 1
9 0 7
2 3 0
7 1 2
... ......

26
icle has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSE.2018.288495
7
…
…
numeric vector)
Input layer
C1: 4 feature maps
S1: 4 feature maps
C2: 6 feature maps
S2: 6 feature maps
Convolutional layer
Convolutional
Subsampling
Output
layer
Dense layer
Output is extracted
features
ReturnStatement
return
ArrayType
String[]
Variable
listVar
Method
toArray
ArrayCreation
new
ArrayType
String[]
NumberLiteral
“0”
0 0 0 0 0 00
0 0 0 0 0 00
2 6 9 ...
...
...
...
...
...
...
...
...
...
...
...
...
...
8 4 1
9 0 7
2 3 0
7 1 2
... ......

27
Inconsistency detection
N
B
findField
{
for(…)
{
…

28
N
B
findField
{
for(…)
{
…
Adjacent
Methods

=
29
N
B
findField
{
for(…)
{
…

30
=
True
False
The method is likely to have
a consistent name.
The method name could be
inconsistent with the implementation.
Suggest a new name.→

33
Name Suggestion
Four ranking strategies:
Suggestion = Sorting similar imp.

33
Name Suggestion
R1: Sort names based on distance,
don’t care identical names.

33
Name Suggestion
R2: Group identical names ﬁrst,
sort groups based on size.

33
Name Suggestion
sort groups based on avg. distance.

33
Name Suggestion
sort groups based on avg. distance.
R4: Same with R3, but penalize groups
with size=1.

35
Research Questions
RQ1: Inconsistency Identification
RQ2: Suggestion Precision
RQ3: Comparative Study
RQ4: Live Study

35
Research Questions
RQ4: Live Study
}Training/testing data from
open-source projects.

35
Research Questions
RQ4: Live Study
Comparing with an approach* with
based on a convolutional attention network.
→
[*] M. Allamanis, H. Peng, and C. Sutton, “A convolutional attention network for extreme summarization of source
code,” in Proceedings of the 33nd International Conference on Machine Learning. JMLR.org, 2016, pp. 2091–2100.

35
Research Questions
RQ4: Live Study
Comparing with an approach* with
based on a convolutional attention network.
→
Submitting our suggestion results as pull-requests
to open-source projects.
→
[*] M. Allamanis, H. Peng, and C. Sutton, “A convolutional attention network for extreme summarization of source
code,” in Proceedings of the 33nd International Conference on Machine Learning. JMLR.org, 2016, pp. 2091–2100.

36
Training/Testing Set
Total: 430 projects

36
Total: 430 projects
Training Data
2,116,413 methods

37
→
Testing
2,805 methods
(name pairs + bodies)

38
RQ1: Inconsistency Identiﬁcation
# of neighbors to look up k=1 5 10 30
Inconsistent
(%)
Precision 56.8 53.7 53.3 49.9
Recall 84.5 55.9 46.7 28.8
F1 67.9 54.8 49.7 36.5
Consistent
(%)
Precision 72.0 55.9 54.2 51.4
Recall 38.2 53.7 60.7 72.2
F1 49.9 54.8 57.3 60.0
Accuracy (%) 60.9 54.8 53.8 50.9
→

39
RQ2: Name Suggestion
Accuracy (%)
k=thr k=10
R1 R2 R3 R4
First Token
thr=1 23.4 23.2 23.0 24.1
thr=5 35.7 39.4 39.4 39.7
Full Name
thr=1 10.7 11.0 10.9 10.9
thr=5 17.0 18.7 19.0 19.2

40
RQ3: Comparison — Name Suggestion
Accuracy (%)
First Token Full Name
thr=1 thr=5 thr=1 thr=5
R1 36.4 47.2 16.5 22.9
R2 34.8 50.2 17.0 25.4
R3 34.7 50.3 16.9 25.5
R4 35.4 50.5 16.0 25.7
conv_attention 22.3 33.6 0.3 0.6
copy_attention 23.5 44.7 0.4 1.1
state-of-the-art
}

41
Training Data
RQ4: Live Study — Setup

41
Training Data
10%

42
Training Data
10%

42
Training Data Identify inconsistent names
and suggest new names
(sampled 100 cases).
10%

42
Create a pull request
10%

42
Create a pull request
Ask a maintainer to refactor
the method names
10%

43
RQ4: Live Study
Agreed Agreed but not ﬁxed
Disagreed Ignored Total
Merged Approved Improved Cannot Won’t
40 26 4 1 2 9 18 100

43
RQ4: Live Study
40 26 4 1 2 9 18 100
Half of them are public methods.

43
RQ4: Live Study
40 26 4 1 2 9 18 100
Developer feedback includes

43
RQ4: Live Study
40 26 4 1 2 9 18 100
* It should follow project-speciﬁc naming conventions.

43
RQ4: Live Study
40 26 4 1 2 9 18 100
* Some method names should consider class names.
e.g., In “XXXBuilder”, many methods cannot be named as “build()”
even though they return “XXXBuilder” objects.

44
Summary
X
RQ4: Live Study
40 26 4 1 2 9 18 100
* Some method names should consider class names.
e.g., In “XXXBuilder”, many methods cannot be named as “build()”
even though they return “XXXBuilder” objects.
X
RQ3: Comparison
Accuracy (%)
First Token Full Name
thr=1 thr=5 thr=1 thr=5
R1 36.4 47.2 16.5 22.9
R2 34.8 50.2 17.0 25.4
R3 34.7 50.3 16.9 25.5
R4 35.4 50.5 16.0 25.7
conv_attention 22.3 33.6 0.3 0.6
copy_attention 23.5 44.7 0.4 1.1
X
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSE.2018.2884955, IEEE
7
…
…
numeric vector)
Input layer
C1: 4 feature maps
S1: 4 feature maps
C2: 6 feature maps
S2: 6 feature maps
Convolutional layer
Convolutional
Subsampling
Output
layer
Dense layer
Output is extracted
features
ReturnStatement
return
ArrayType
String[]
Variable
listVar
Method
toArray
ArrayCreation
new
ArrayType
String[]
NumberLiteral
“0”
0 0 0 0 0 00
0 0 0 0 0 00
the first subsampling layer, and S2 is the second one. The output of dense layer is considered as extracted features of code
fragments and will be used to do clustering.
2.4.4 Code Patterns Mining
Although violations can be parsed and converted into two-
dimensional numeric vectors, it is still challenging to mine
code patterns given that noisy information (e.g., specific
meaningless identifiers) can interfere with identifying sim-
ilar violations. Deep learning has recently been shown
promising in various software engineering tasks [18], [47],
[49]. In particular, it offers a major advantage of requiring
less prior knowledge and human effort in feature design for
machine learning applications. Consequently, our method is
designed to deeply learn discriminating features for mining
code patterns of violations. We leverage CNNs to perform
deep learning of violation features with embedded viola-
tions, and also use X-means clustering algorithm to cluster
violations with learned features.
Feature learning with CNNs
Figure 8 shows the CNNs architecture for learning violation
features. The input is two-dimensional numeric vectors
of preprocessed violations. The alternating local-connected
convolutional and subsampling layers are used to capture
the local features of violations. The dense layer compresses
all local features captured by former layers. We select the
output of the dense layer as the learned violation features
to cluster violations. Note that our approach uses CNNs to
of violations from clustered similar code fragments of viola-
tions to show patterns clearly. Note that, the whole process
of mining patterns is automated.
2.5 Mining Common Fix Patterns
Our goal in this step is to summarize how a violation
is resolved by developers. To achieve this goal, we col-
lect violation fixing changes and proceed to identify their
common fix patterns. The approach of mining common fix
patterns is similar to that of mining common code patterns.
The differences lie in the data collection and tokenization
process. Before describing our approach of mining common
fix patterns, we formalize the definitions of patch and fix
pattern.
2.5.1 Preliminaries
A patch represents a modification carried on a program
source code to repair the program which was brought to
an erroneous state at runtime. A patch thus captures some
knowledge on modification behavior, and similar patches
may be associated with similar behavioral changes.
Definition 4. Patch (P): A patch is a pair of source code
fragments, one representing a buggy version and another
as its updated (i.e., bug-fixing) version. In the traditional
GNU diff representation of patches, the buggy version is
2 6 9 ...
...
...
...
...
...
...
...
...
...
...
...
...
...
8 4 1
9 0 7
2 3 0
7 1 2
... ......
X
Making the name consistent is NOT easy
https://www.itworld.com/article/2833265/
don-t-go-into-programming-if-you-don-t-
have-a-good-thesaurus.html

45
https://github.com/SerVal-DTF/debug-method-name
Tool and Data

46
https://www.darkrsw.net http://wwwen.uni.lu/snt/
research/serval
Hire me! Université du Luxembourg
1.1 - logotype of the University
of Luxembourg
The logotype may not be altered under any
circumstances.
It is to be used like this for all communication mediums.
Université du Luxembourg © 03/2013
3.1 - the Interdisciplinary Centre for
Security Reliability and Trust
The SnT uses its own logo. It is used on all external
communication tools in combination with the UL logo.
Design guidelines are available at SnT.
Hiring

Learning to Spot and Refactor Inconsistent Method Names

Learning to Spot and Refactor Inconsistent Method Names

Recommended

More Related Content

What's hot

What's hot(20)

Similar to Learning to Spot and Refactor Inconsistent Method Names

Similar to Learning to Spot and Refactor Inconsistent Method Names(20)

Recently uploaded

Recently uploaded(20)

Learning to Spot and Refactor Inconsistent Method Names