Talk on the Workshop of Refactoring Tools

1. What Programmers Say About
Refactoring Tools?
An Empirical Investigation of StackOverflow
Gustavo Pinto, Fernando Kamei
Federal University of Pernambuco
{ghlp, fkk}@cin.ufpe.br

2. Researchers
Programmers

3. Programmers
Researchers
What they think about?

4. Programmers
Researchers
What they think about?

5. Programmers
Researchers
What they think about?

6. 1.5M Users
3.4M Questions
6.8M Answers
40GB of data
A TON of information
(with some quality)

7. 1.5M Users
3.4M Questions
6.8M Answers
40GB of data
A TON of information
(with some quality)
But, what about
Refactoring Tools?

8. 3.4M Questions
Filter questions by “refactoring”
and “tools” on the body, or
subject or tag name.
Store the results in a
single MySQL table.
754 Q
>2K A
from July 31, 2008
to July 31, 2012:
●
●
●
324 Questions
1,135 Answers
1,254 Users
The final data!
Manual filter to remove
false positives

9. 3.4M Questions
Filter questions by “refactoring”
and “tools” key-word, on the
body, or subject or tag name.
Store the results in a
single MySQL table.
754 Q
>2K A
from July 31, 2008
to July 31, 2012:
●
●
●
324 Questions
1,135 Answers
1,254 Users
The final data!
Manual filter to remove
false positives

10. 3.4M Questions
Filter questions by “refactoring”
and “tools” key-word, on the
body, or subject or tag name.
Store the results in a
single MySQL table.
754 Q
>2K A
from July 31, 2008
to July 31, 2012:
●
●
●
324 Questions
1,135 Answers
1,254 Users
2x
The final data!
Manual filter to remove
false positives

11. Research Questions
● RQ1: What are the most desirable features in
refactoring tools?
● RQ2: What are the barriers to adoption of refactoring
tools?
● RQ3: How is interest in refactoring tools related to
programmer expertise?
● RQ4: Does interest in refactoring tools increase over
the years?

12. RQ1: What are the most desirable features in refactoring tools?
1. Refactoring for dynamic languages (78 questions ≈ 24.72%)
a. PHP
» 29%
“I am looking for a python tool that will allow
b. JavaScript
» 23%
me to specifically rename and move
c. Python
» 23%
modules.”
d. Ruby
» 11%
StackOverflow Question #9424189
e. Perl
» 9%
f. R
» 1%
g. Perl
» 1%
h. Matlab
» 1%

13. RQ1: What are the most desirable features in refactoring tools?
1. Refactoring for dynamic languages (78 questions ≈ 24.72%)
2. Refactoring recommendations (45 questions ≈ 13.88%)
“I would prefer a tool that just makes suggestions about possible refactorings: names
the refactoring, optionally provides a short description of it (great for learning purposes),
highlights the code section and lets me do the refactoring myself”
StackOverflow Question #785667

14. RQ1: What are the most desirable features in refactoring tools?
1. Refactoring for dynamic languages (78 questions ≈ 24.72%)
2. Refactoring recommendations (45 questions ≈ 13.88%)
3. Refactoring for databases (12 questions ≈ 3.71%)
a. Automatic simplification of SQL
b. Refactoring table or views structure
c. Fixing SQL vulnerabilities
“I would like to rename a table column in MySql and also have the name updated in any triggers,
stored procs, etc. that reference the column. [...] If a tool exists that can perform such a
refactoring, it would be great if it could generate an SQL script that applies these changes.”
StackOverflow Question #1644271

15. RQ1: What are the most desirable features in refactoring tools?
1.
2.
3.
4.
Refactoring for dynamic languages (78 questions ≈ 24.72%)
Refactoring recommendations (45 questions ≈ 13.88%)
Refactoring for databases (12 questions ≈ 3.71%)
Multi-language Refactorings (9 questions ≈ 2.77%)
a. All questions related to JVM languages
“I have inherited significant amounts of Groovy code, and I have found it difficult to maintain [..]. I
have tried to manually convert some pieces to Java, it's been a pain. Are you aware of any tools
or plugins that help with this conversion?”
StackOverflow Question #5302103

16. RQ2: What are the barriers to adoption of refactoring tools?
In order of interest:
1. Usability problems (52 questions ≈ 16%)
1.1. Unknown tool (54%)
1.2. Difficult to learn (13%)
1.3. Difficult to use (8%)
2. Lack of trust (39 questions ≈ 12%)

17. RQ2: What are the barriers to adoption of refactoring tools?
In order of interest:
1. Usability problems (16% of Q)
1.1. Unknown tool (54%)
1.2. Difficult to learn (13%)
1.3. Difficult to use (8%)
33% are from novice users
64%
49%
What refactoring tool
do you use to do X in
language Z?

18. RQ2: What are the barriers to adoption of refactoring tools?
In order of interest:
1. Usability problems (16% of Q)
1.1. Unknown tool (54%)
1.2. Difficult to learn (13%)
1.3. Difficult to use (8%)
Barriers or lack of experience?
33% are from novice users
64%
49%
What refactoring tool
do you use to do X in
language Z?

19. RQ2: What are the barriers to adoption of refactoring tools?
In order of interest:
1. Usability problems (16% of Q)
1.1. Unknown tool (54%)
1.2. Difficult to learn (13%)
1.3. Difficult to use (8%)
2. Lack of trust (12 % of Q)
Sometimes the tool will change
the meaning of your code,
without you expecting it.
StackOverflow Question #1372059

20. RQ2: What are the barriers to adoption of refactoring tools?
In order of interest:
1. Usability problems (16% of Q)
1.1. Unknown tool (54%)
1.2. Difficult to learn (13%)
1.3. Difficult to use (8%)
2. Lack of trust (12 % of Q)
“It is not good to decrease readability in
favor of atomicity”
StackOverflow Question #5302103
Sometimes the tool will change
the meaning of your code,
without you expecting it.
StackOverflow Question #1372059

21. RQ3: How is interest in refactoring tools related to programmer
expertise?
Or.. do only hackers talk about refactoring tools?

22. RQ3: How is interest in refactoring tools related to programmer
expertise?
Or.. do only hackers talk about refactoring tools?
Q: 39% comes from novices
A: 41% comes from seniors
Novices
Seniors

23. RQ3: How is interest in refactoring tools related to programmer
expertise?
Or.. do only hackers talk about refactoring tools?
Q: 39% comes from novices
A: 41% comes from seniors
Refactoring tool is a topic that
interest all kind of users.
Novices
Seniors

24. RQ3: How is interest in refactoring tools related to programmer
expertise?
Or.. do only hackers talk about refactoring tools?
Q: 39% comes from novices
A: 41% comes from seniors
Novices
Seniors
Also, we ran a Person
Correlation between reputation x
questions and reputation x
answers, but no significant
correlation was found.

25. RQ4: Does interest in refactoring tools increase over the years?
Questions per month
● Median: 6
● Mean: 6.75
● SD: 3.38
● Max: 20
● Min: 1
Answers per month
● Median: 18
● Mean: 23.63
● SD: 16.23
● Max: 73
● Min: 1

26. RQ4: Does interest in refactoring tools increase over the years?
Questions per month
● Median: 6
● Mean: 6.75
● SD: 3.38
● Max: 20
● Min: 1
● Yes, but very little.
Answers per month
● Median: 18
● Mean: 23.63
● SD: 16.23
● Max: 73
● Min: 1

27. RQ4: Does interest in refactoring tools increase over the years?
Questions per month
● Median: 6
● Mean: 6.75
● SD: 3.38
● Max: 20
● Min: 1
● Yes, but very little.
● An increment of 0.56 questions per trimester.
● A decrement of -8.13 answers per trimester.
Answers per month
● Median: 18
● Mean: 23.63
● SD: 16.23
● Max: 73
● Min: 1

28. RQ4: Does interest in refactoring tools increase over the years?
Questions per month
● Median: 6
● Mean: 6.75
● SD: 3.38
● Max: 20
● Min: 1
-0.43 without outliers
● Yes, but very little.
● An increment of 0.56 questions per trimester.
● A decrement of -8.13 answers per trimester.
Answers per month
● Median: 18
● Mean: 23.63
● SD: 16.23
● Max: 73
● Min: 1

29. Conclusions
● We provided a set of desired features, as well as common
cause for barriers adoption.
○ Insights for researchers and tool vendors
● We observed that novice users produce more questions, and
seniors provide the answers.
○ Also, most of the barriers were reported by novice users
● We identified a relative small interest in refactoring tools
○ Mean of 6 questions / month, sd of 4.45

30. What Programmers Say About
Refactoring Tools?
An Empirical Investigation of StackOverflow
Gustavo Pinto, Fernando Kamei
Federal University of Pernambuco
{ghlp, fkk}@cin.ufpe.br