Miryung
Kim1,
Dongxiang
Cai2,
Sunghun
Kim2
The
University
of
Texas
at
Austin1

Hong
Kong
University
of
Science
&
Technolog...
  It
is
believed
that
refactoring
improves
   software
quality
and
maintainability.


  The
role
of
refactoring
has
not
...
    We
studied
the
relationships
between
API‐level
     refactorings,
bug
fixes,
the
time
taken
to
fix
     bugs,
and
relea...
  Motivation
&
Related
Work

  Study
Approach
Overview
  Research
Questions
and
Results

  Limitations
  Conclusions

  Refactoring
improves
software
quality
and
   maintainability
[Opdyke,
Griswold,
Mens,
etc.]

  A
lack
of
refactoring
i...
  The
number
of
bug
reports
increases
after
   refactorings
[Weißgerber
&
Diehl]

  Incomplete
or
inconsistent
refactori...
  Refactoring
tools
are
buggy
[Daniel
et
al.]
  Programmers
do
not
leverage
refactoring
   tools
effectively
[Murphy‐Hill...
  Motivation
  Study
Approach
Overview
  Research
Questions
and
Results

  Limitations
  Related
Work
  Conclusions

  Q1:
Are
there
more
bug
fixes
after
API‐level
   refactorings?

  Q2:
Are
API‐level
refactorings
correlated
to
   bug
fix...
API‐level
     Refactoring
                       
Fix
Revision

                                        [ICSE
2007,

    ...
Methods
                        Change
           Add
                   Delete
                                          ...
API‐level
   Refactoring
                Rename,
move,
and
signature
changes

  Reconstruction
              at
or
above
t...
Boo.mB(long)
=>
Baz.mB(int)
Foo.mA()
=>
Foo.mA(int)
               Foo.mB()
=>
Foo.mB(int)
   r1
   r2
      r3
   r4
    ...
Boo.mB(long)
=>
Baz.mB(int)
Foo.mA()
=>
Foo.mA(int)
               Foo.mB()
=>
Foo.mB(int)
   r1
   r2
      r3
   r4
    ...

Fix
Revision

      Use
heuristics
to
identify
bug
fix
revisions

   [ICSE
2007,

                           e.g.,
[bug]
[...

Introducing
       Change
               Use
SVN
blame
(diff)
to
track
down

  [MSR
2005,

               the
changes
that...
Bug
life
time
Bug‐Introduction
          Bug‐Fix
   r1
    r2
   r3
    r4
    r5
     r6
     r7
    r8
         Ref
    ...
Change
Distilling
            Map
line‐level
location
to
a
method
level
location

    [TSE
2007,

              Filter
out...
Eclipse
JDT
      jEdit
             Columba
                  core
Type
             IDE
              Text
editor
      ...
GT
               source
               #
    Prec.

   recall
    API
      14
 API‐level
refactoring
detection

   14
  ...
  Motivation
and
Related
Work
  Study
Approach
Overview
  Research
Questions
and
Results

  Limitations
  Conclusions

  Study
Method
1:
We
measure
a
fix
rate
(#
fix
 revisions
/
K)
within
K
sliding
window
after
 each
refactoring
revision.


...
  Study
Method
1:
We
measure
a
fix
rate
(#
fix
 revisions
/
K)
within
K
sliding
window
after
 each
refactoring
revision.


...
  Study
Method
1:
We
measure
a
fix
rate
(#
fix
 revisions
/
K)
within
K
sliding
window
after
 each
refactoring
revision.


...
  Study
Method
1:
We
measure
a
fix
rate
(#
fix
 revisions
/
K)
within
K
sliding
window
after
 each
refactoring
revision.


...
  Study
Method
1:
We
measure
a
fix
rate
(#
fix
  revisions
/
K)
within
K
sliding
window
after
  each
refactoring
revision.
...
  Results:
Fix
rates
increase
after
refactorings
  at
a
revision
level

                                              ...
  Results:
fix
rates
at
the
method
level
also
            increase
after
refactorings
Fix
rate
at
a
method
level
0.40%
   ...
20
revision
windows
             Ref
              Fix
 Fix
In
Eclipse,
208
refactoring
revisions
are
followed
by
at
least...
20
revision
windows
        Ref
              Fix
 Fix
We
randomly
sampled
50
revision
pairs
and
manually
               i...
R
          F
                            Types
                                Frequency
1.

 To
fix
several
related
bugs,...
  Study
Method
1:
We
measure
the
extent
of
 revisions
that
include
both
refactorings
and
 bug
fixes
at
the
same
time.

  ...
Probability
of
fixes
and
refactorings
at
the
revision
level


Project
         P(F)
     P(R)
    P(R|F)
 P(R|not
F)
 P(F|R...
Probability
of
fixes
and
refactorings
at
the
revision
level


Project
         P(F)
     P(R)
    P(R|F)
 P(R|not
F)
 P(F|R...
  Results:
 #
of
Ref‐revisions
that
were
            #
of
Non‐ref
revisions
that
were
 related
to
fix
within
(0,K]
       ...
  Study
Method:
We
estimate
the
time
taken
 to
fix
bugs
introduced
and
resolved
near
the
 refactoring
timing.

           ...
  Study
Method:
We
estimate
the
time
taken
 to
fix
bugs
introduced
and
resolved
near
the
 refactoring
timing.

           ...
  Study
Method:
We
estimate
the
time
taken
 to
fix
bugs
introduced
and
resolved
near
the
 refactoring
timing.

           ...
  Study
Method:
We
estimate
the
time
taken
 to
fix
bugs
introduced
and
resolved
near
the
 refactoring
timing.

           ...
  Study
Method:
We
estimate
the
time
taken
 to
fix
bugs
introduced
and
resolved
near
the
 refactoring
timing.

           ...
  Results:
average
fix

                K=100
time
(days)
      25
                                 120
                  ...
  Study
Method:
We
measure
refactoring
 rates
and
fix
rates
before
major
releases.

      K
revision
windows
     Ref
    ...
  Results
                             30%
                    Eclipse
JDT
                             25%
         refa...
  Study
intra‐method
refactorings
and
   complex
refactorings
(e.g.,
use
RefFinder
   [Prete
et
al.])
  Account
for
the
...
  API‐level
refactoring
and
bug
fixes
are
   correlated
temporally
and
spatially.

  API‐level
refactoring
serves
the
rol...
  This
research
is
in
part
supported
by
National
   Science
Foundation,
CCF‐1043810.

  We
thank
anonymous
reviewers
and...
Upcoming SlideShare
Loading in …5
×

ICSE 2011 Research Paper on Refactorings and Bug Fixes

1,871 views
1,788 views

Published on

An Empirical Investigation into the Role of API-Level Refactoring during Software Evolution, Miryung Kim, Dongxiang Cai, Sunghun Kim, ICSE' 11: Proceedings of the 2011 ACM and IEEE 33rd International Conference on Software Engineering

1 Comment
4 Likes
Statistics
Notes
No Downloads
Views
Total views
1,871
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
1
Likes
4
Embeds 0
No embeds

No notes for slide

ICSE 2011 Research Paper on Refactorings and Bug Fixes

  1. 1. Miryung
Kim1,
Dongxiang
Cai2,
Sunghun
Kim2
The
University
of
Texas
at
Austin1

Hong
Kong
University
of
Science
&
Technology2

  2. 2.   It
is
believed
that
refactoring
improves
 software
quality
and
maintainability.


  The
role
of
refactoring
has
not
been
 systematically
investigated
using
fine‐ grained
revision
history
at
the
method
level.

  3. 3.   We
studied
the
relationships
between
API‐level
 refactorings,
bug
fixes,
the
time
taken
to
fix
 bugs,
and
release
cycles.

 •  The
number
of
bug‐fixes
increases
after
API‐level
 refactorings.
 •  A
large
number
of
refactoring
revisions
include
 bug
fixes
or
are
related
to
later
fixes.

 •  Bug
fix
time
tends
to
be
shorter
after
than
before.

 •  API‐level
refactorings
occur
more
frequently
 before
major
releases
than
after.


  4. 4.   Motivation
&
Related
Work

  Study
Approach
Overview
  Research
Questions
and
Results

  Limitations
  Conclusions

  5. 5.   Refactoring
improves
software
quality
and
 maintainability
[Opdyke,
Griswold,
Mens,
etc.]

  A
lack
of
refactoring
incurs
technical
debt.
 [Cunningham,
Lehman]
  Refactor
mercilessly
[Beck,
eXtreme
 Programming]
  Productivity
improves
after
re‐architecting
a
 system
[Carriere
et
al.]

  Defect
density
decreases
after
refactoring
 [Ratzinger
et
al.]


  6. 6.   The
number
of
bug
reports
increases
after
 refactorings
[Weißgerber
&
Diehl]

  Incomplete
or
inconsistent
refactorings
cause
 bugs
[Görg
and
Weißgeber]
  Code
churns
are
correlated
with
defect
 density
[Nagappan
&
Ball]

  7. 7.   Refactoring
tools
are
buggy
[Daniel
et
al.]
  Programmers
do
not
leverage
refactoring
 tools
effectively
[Murphy‐Hill
et
al.]
  API‐level
refactorings
break
client
code
[Dig
 et
al.]

  8. 8.   Motivation
  Study
Approach
Overview
  Research
Questions
and
Results

  Limitations
  Related
Work
  Conclusions

  9. 9.   Q1:
Are
there
more
bug
fixes
after
API‐level
 refactorings?

  Q2:
Are
API‐level
refactorings
correlated
to
 bug
fixes?
  Q3:
Do
API‐level
refactorings
reduce
the
time
 taken
to
fix
bugs?

  Q4:
Are
there
relatively
fewer
API‐level
 refactorings
before
release
dates?


  10. 10. API‐level
 Refactoring
 
Fix
Revision

 [ICSE
2007,

 Reconstruction
 S.
Kim
et
al.]

 [ICSE
2007,
M.
Kim
et
al.]

 Identify
Refactoring

 Identify
 Revisions
 
Fix
Revisions
 
Introducing
 Change
Distilling
 Change
 [TSE
2007,

 [MSR
2005,

 B.
Fluri
et
al.]

 Sliwerski

et
al.]

Identify
Bug
Life
Time
 Map
line‐level
changes
 to
methods


  11. 11. Methods
 Change
 Add
 Delete
 Revisions
 r1
 r2
 r3
 r4
 r5
 r6
 r7
 r8

  12. 12. API‐level
 Refactoring
 Rename,
move,
and
signature
changes

 Reconstruction
 at
or
above
the
method
header
level

[ICSE
2007,
M.
Kim
et
al.]

 Foo.mA(int) Foo.mA() Foo.mB(int) Foo.mB() Foo.mC() Foo.mC() Bar.mA(long) Boo.mA(long) Baz.mA(int) Boo.mB(long) Baz.mB(int)
  13. 13. Boo.mB(long)
=>
Baz.mB(int)
Foo.mA()
=>
Foo.mA(int)
 Foo.mB()
=>
Foo.mB(int)
 r1
 r2
 r3
 r4
 r5
 r6
 r7
 r8

  14. 14. Boo.mB(long)
=>
Baz.mB(int)
Foo.mA()
=>
Foo.mA(int)
 Foo.mB()
=>
Foo.mB(int)
 r1
 r2
 r3
 r4
 r5
 r6
 r7
 r8
 Ref
 Ref

  15. 15. 
Fix
Revision

 Use
heuristics
to
identify
bug
fix
revisions

 [ICSE
2007,

 e.g.,
[bug]
[id
#],
[fixed]

 S.
Kim
et
al.]

r1
 r2
 r3
 r4
 r5
 r6
 r7
 r8
 Ref
 Fix
 Ref/Fix
 Fix

  16. 16. 
Introducing
 Change
 Use
SVN
blame
(diff)
to
track
down

 [MSR
2005,

 the
changes
that
led
to
fixes.

Sliwerski

et
al.]

 Bug‐introducing
changes

r1
 r2
 r3
 r4
 r5
 r6
 r7
 r8
 Ref
 Fix
 Ref/Fix
 Fix

  17. 17. Bug
life
time
Bug‐Introduction
 Bug‐Fix
 r1
 r2
 r3
 r4
 r5
 r6
 r7
 r8
 Ref
 Fix
 Ref/Fix
 Fix

  18. 18. Change
Distilling
 Map
line‐level
location
to
a
method
level
location

 [TSE
2007,

 Filter
out
meaningless
changes.

 B.
Fluri
et
al.]

 r1
 r2
 r3
 r4
 r5
 r6
 r7
 r8
 Ref
 Fix
 Ref/Fix
 Fix

  19. 19. Eclipse
JDT
 jEdit
 Columba
 core
Type
 IDE
 Text
editor
 Email
client
Period
 June
5th,
2001
to
 Sep
2,
2001
to
 July
9th,
2006
to
 May
16th
2007
 Apr,
2009
 Jan
31st
2010
#
revisions
 15000
 11102
 421
#
of
API‐level
 6755
 3557
 424

refactorings
#
of
bug
fix
 3752
 1073
 150
revisions
#
of
refactoring
 1089
 423
 36
revisions

  20. 20. GT
 source
 #
 Prec.

 recall
 API
 14
 API‐level
refactoring
detection

 14
 0.93
 0.93
Refactoring

 [M.
Kim
et
al.
]

 revision
 Inspection
of
change
logs
only
 2
 0.50
 0.07
 Inspection
of
bug
reports
only
 5
 0.75
 0.21
 Fix

 62
 Fix‐revision
detection
[S.
Kim
 49
 0.96
 0.76
 Revision

 et
al.]

 Inspection
of
change
logs
only
 55
 0.92
 0.81
 Inspection
of
bug
reports
only
 49
 0.94
 0.74
 Automated
techniques
vs.
Manual
Inspection

  21. 21.   Motivation
and
Related
Work
  Study
Approach
Overview
  Research
Questions
and
Results

  Limitations
  Conclusions

  22. 22.   Study
Method
1:
We
measure
a
fix
rate
(#
fix
 revisions
/
K)
within
K
sliding
window
after
 each
refactoring
revision.


 K
revision
windows
 Ref
 Ref
 Fix
 Ref
 Fix
 Fix
 Ref
 Fix

  23. 23.   Study
Method
1:
We
measure
a
fix
rate
(#
fix
 revisions
/
K)
within
K
sliding
window
after
 each
refactoring
revision.


 K
revision
windows
 Ref
 Ref
 Fix
 Ref
 Fix
 Fix
 Ref
 Fix

  24. 24.   Study
Method
1:
We
measure
a
fix
rate
(#
fix
 revisions
/
K)
within
K
sliding
window
after
 each
refactoring
revision.


 K
revision
windows
 Ref
 Ref
 Fix
 Ref
 Fix
 Fix
 Ref
 Fix

  25. 25.   Study
Method
1:
We
measure
a
fix
rate
(#
fix
 revisions
/
K)
within
K
sliding
window
after
 each
refactoring
revision.


 K
revision
windows
 Ref
 Ref
 Fix
 Ref
 Fix
 Fix
 Ref
 Fix

  26. 26.   Study
Method
1:
We
measure
a
fix
rate
(#
fix
 revisions
/
K)
within
K
sliding
window
after
 each
refactoring
revision.

 K
revision
windows
 Ref
 Ref
 Fix
 Ref
 Fix
 Fix
 Ref
 Fix
  Study
Method
2:
We
measure
a
fix
rate
at
 the
method
level
 M.foo()=>M.bar()
 Ref
 Fix
 Fix

  27. 27.   Results:
Fix
rates
increase
after
refactorings
 at
a
revision
level

     average

   fix
rate
           Window
size
(K)
 Refactoring
timing

  28. 28.   Results:
fix
rates
at
the
method
level
also
 increase
after
refactorings
Fix
rate
at
a
method
level
0.40%
 1.00%
 8.00%
0.30%
 0.80%
 6.00%
 0.60%
0.20%
 4.00%
 0.40%
0.10%
 0.20%
 2.00%
0.00%
 0.00%
 0.00%
 ‐30
 ‐20
 ‐10
 0
 10
 20
 30
 ‐30
 ‐20
 ‐10
 0
 10
 20
 30
 ‐30
 ‐20
 ‐10
 0
 10
 20
 30
 Refactoring
timing
 JDT
 jEdit
 Columba
 There
is
a
short
term
increase
in
the
number
of
bug
fixes
 after
refactorings

  29. 29. 20
revision
windows
 Ref
 Fix
 Fix
In
Eclipse,
208
refactoring
revisions
are
followed
by
at
least
one
bug
fix
to
the
same
method
location
within
20
revisions.


  30. 30. 20
revision
windows
 Ref
 Fix
 Fix
We
randomly
sampled
50
revision
pairs
and
manually
 inspected
them.


  31. 31. R
 F
 Types
 Frequency
1.

 To
fix
several
related
bugs,
a
developer
first
refactors
 6%
 code.

2.

 Incorrect
refactorings
cause
a
bug,
inducing
a
later
fix.

 6%

3.

 After
a
developer
performs
refactorings,
she
discovers
a
 2%

 new
bug
and
fixes
it.

4
 Refactoring
with
a
bug
fix
is
incomplete.
A
developer
 50%

 applies
a
supplementary
fix
later.
5.

 A
bug
fix
happened
after
a
refactoring
in
the
same
 36%

 location,
but
they
are
not
related.


  32. 32.   Study
Method
1:
We
measure
the
extent
of
 revisions
that
include
both
refactorings
and
 bug
fixes
at
the
same
time.

  Study
Method
2:
We
measure
the
 percentage
refactoring
revisions
that
have
at
 least
one
bug
fix
applied
to
the
same
method
 location
within
K
revisions.




  33. 33. Probability
of
fixes
and
refactorings
at
the
revision
level


Project
 P(F)
 P(R)
 P(R|F)
 P(R|not
F)
 P(F|R)
 P(F|not
R)
 Eclipse
 25%
 7.3%
 12%
 5.7%
 41.5%
 23.7%
 JDT
 JEdit
 9.7%
 3.8%
 11.5%
 3.0%
 29.1%
 8.9%
Columba
 35.7%
 8.6%
 10.7%
 7.4%
 44.4%
 34.8%

  34. 34. Probability
of
fixes
and
refactorings
at
the
revision
level


Project
 P(F)
 P(R)
 P(R|F)
 P(R|not
F)
 P(F|R)
 P(F|not
R)
 Eclipse
 25%
 7.3%
 12%
 5.7%
 41.5%
 23.7%
 JDT
 JEdit
 9.7%
 3.8%
 11.5%
 3.0%
 29.1%
 8.9%
Columba
 35.7%
 8.6%
 10.7%
 7.4%
 44.4%
 34.8%
 Refactorings
and
bug
fixes
often
occur
together
in
the
same
 revisions.



  35. 35.   Results:
 #
of
Ref‐revisions
that
were
 #
of
Non‐ref
revisions
that
were
 related
to
fix
within
(0,K]
 related
to
fix
within
(0,K]
 2245
 (14.0%)
 495
 (32.0%)
 1053
 13847
(86.0%)
 (68.0%)
 K=20
Refactoring
revisions
are
followed
by
related
bug
fixes
more
 often
than
non‐refactoring
revisions.


  36. 36.   Study
Method:
We
estimate
the
time
taken
 to
fix
bugs
introduced
and
resolved
near
the
 refactoring
timing.

 K
revisions
 K
revisions
 K
revisions
 refactoring
 Bug
Introduction
 Bug
Fix
 revision

  37. 37.   Study
Method:
We
estimate
the
time
taken
 to
fix
bugs
introduced
and
resolved
near
the
 refactoring
timing.

 K
revisions
 K
revisions
 K
revisions
 AA
 refactoring
 Bug
Introduction
 Bug
Fix
 revision

  38. 38.   Study
Method:
We
estimate
the
time
taken
 to
fix
bugs
introduced
and
resolved
near
the
 refactoring
timing.

 K
revisions
 K
revisions
 K
revisions
 AA
 BB
 refactoring
 Bug
Introduction
 Bug
Fix
 revision

  39. 39.   Study
Method:
We
estimate
the
time
taken
 to
fix
bugs
introduced
and
resolved
near
the
 refactoring
timing.

 K
revisions
 K
revisions
 K
revisions
 AA
 BB
 BA
 refactoring
 Bug
Introduction
 Bug
Fix
 revision

  40. 40.   Study
Method:
We
estimate
the
time
taken
 to
fix
bugs
introduced
and
resolved
near
the
 refactoring
timing.

 K
revisions
 K
revisions
 K
revisions
 AA
 XB
 BB
 BA
 refactoring
 Bug
Introduction
 Bug
Fix
 revision

  41. 41.   Results:
average
fix

 K=100
time
(days)
 25
 120
 250
 230.2
 19.7
 98.6
 20
 100
 200
 80
 149.6
 15
 150
 60
 42.6
 10
 7.3
 100
 5.6
 40
 4.4
 21.5
 12.2
 38.4
 48.4
 5
 20
 50
 0
 0
 0
 BB
 AA
 XB
 BA
 BB
 AA
 XB
 BA
 BB
 AA
 XB
 BA
 JDT
 jEdit
 Columba
 When
it
comes
to
fixing
bugs
introduced
near
the
time
of
 refactorings,
the
average
fix
time
tends
to
decrease
after
 refactorings

  42. 42.   Study
Method:
We
measure
refactoring
 rates
and
fix
rates
before
major
releases.

 K
revision
windows
 Ref
 Ref
 Fix
 Ref
 Fix
 Fix
 Ref
 Fix
 Major
Release

  43. 43.   Results
 30%
 Eclipse
JDT
 25%
 refactoring
rate 20%
 15%
 10%
 5%
 0%
 K=‐20
 K=20
 K=‐30
 K=30
 K=‐40
 K=40
 window
size
(K)There
are
more
refactorings
prior
to
major
version
releases.


  44. 44.   Study
intra‐method
refactorings
and
 complex
refactorings
(e.g.,
use
RefFinder
 [Prete
et
al.])
  Account
for
the
size
of
edits
and
the
number
 of
refactorings

  Phase
&
activity
level

  Micro‐commit
practices

  45. 45.   API‐level
refactoring
and
bug
fixes
are
 correlated
temporally
and
spatially.

  API‐level
refactoring
serves
the
role
of
both
 facilitating
bug
fixes
and
introducing
bugs.


  API‐level
refactoring
occurs
as
a
part
of
 behavior
correcting
edits
 Refactoring
engines
must
support
safe
application
of

 refactorings
and
semantic
changes
together.

 The
result
calls
for
an
in‐depth
study
on
the
cost
and
 benefit
of
refactorings.


  46. 46.   This
research
is
in
part
supported
by
National
 Science
Foundation,
CCF‐1043810.

  We
thank
anonymous
reviewers
and
UIUC
 reading
group
for
their
comments.



×