Paper presented at SANER 2017

1. Extracting Executable
Transformations from
Distilled Code Changes
Reinout Stevens
resteven@vub.ac.be
@ReinoutStevens
Coen De Roover
cderoove@vub.ac.be
@oniroi
1

2. Context: Distilled Code Changes
B. Fluri, M. Würsch, M. Pinzger, and H. C. Gall.
Change distilling: Tree differencing for fine-grained source code change extraction.
Transactions on Software Engineering, 2007.
2
1. update(0,1)
2. move(int y = 0;)
3. insert(public int foo()…)
4. delete(int z = 0;)
Output:
00 public class Example {
01 public Integer run(Integer x) {
02 return x;
03 }
04
05 public void test() {
06 int x = 0;
07 int y = 0;
08 int z = 0;
09 run(x);
10 }
11 }
00 public class Example {
01 public Integer run(Integer x) {
02 int y = 0;
03 return x;
04 }
05
06 public int foo() {
07 return 42;
08 }
09
10 public void test() {
11 int x = 1;
12 run(x);
13 }
14 }
Insert
Delete
Move
Update
00 public class Example {
01 public Integer run(Integer x) {
02 return x;
03 }
04
05 public void test() {
06 int x = 0;
07 int y = 0;
08 int z = 0;
09 run(x);
10 }
11 }
00 public class Example {
01 public Integer run(Integer x) {
02 int y = 0;
03 return x;
04 }
05
06 public int foo() {
07 return 42;
08 }
09
10 public void test() {
11 int x = 1;
12 run(x);
13 }
14 }
Revision 1 Revision 2

3. Goal: Extracting Executable Transformations
3
Insert Insert Move Delete Update Insert Move Delete
Delete Insert Insert Move Insert Delete Update Delete
Update Move Insert Insert Update Update Move …

4. Naive Approach: Logic Queries over Changes
public class Example {
int x = 0;
}
public class Example {
int y = 0;
}
1. update(“x”, “y”)
1 (defn rename-field [changes]
2 (run* [?update]
3 (member ?update changes)
4 (change|update ?update)
5 (change|update-original ?update ?val|source)
6 (ast :SimpleName ?val|source)
7 (ast-parent ?val|source ?parent|source)
8 (ast :VariableDeclarationFragment ?parent|source)
9 (update-newval ?update ?val|target)
0 (ast :SimpleName ?val|target)
1 (name-name|different ?val|source ?val|target))
4
Revision 1 Revision 2
changes
look for an update
that modifies a field
its name

5. 1. delete(“int x = 0;”)
2. insert(“int aLongerName = 0;”)
1 (defn rename-field [changes]
2 (run* [?insert ?delete]
3 (fresh [?insert|source’ ?delete|original ?i-name ?d-name]
4 (member ?delete changes)
5 (member ?insert changes)
6 (== ?sequence (list ?insert ?delete))
7 (change|insert ?insert)
8 (change|delete ?delete)
9 (insert-node ?insert ?insert|source’)
0 (ast :VariableDeclarationFragment ?insert|source’)
1 (delete-node ?delete ?delete|original)
2 (ast :VariableDeclarationFragment ?delete|source)
3 (has :name ?insert|source’ ?i-name)
4 (has :name ?delete|source ?d-name)
5 (name-name|different ?i-name ?d-name))) 5
public class Example {
int x = 0;
}
public class Example {
int aLongerName = 0;
}
changes
Revision 1 Revision 2
look for a
delete and insert
that introduce a field
and remove a field with
a different name
Naive Approach: Logic Queries over Changes

6. Problem: Change Equivalence
1. update(“x”, “y”)
1. delete(“int x = 0;”)
2. insert(“int y = 0;”)
1. insert(“int y = 0;”)
2. delete(“int x = 0;”)
Possible
Changes Sequences
public class Example {
int x = 0;
}
Revision 1
public class Example {
int y = 0;
}
Revision 2
6
One change distiller may produce different change sequences for
the same source code transformation in different commits:
• different change types
• different length
• different subjects of changes

7. 7
Problem: Change Equivalence
or
or
…

8. Problem Summary
Multiple change sequences implement the same source
code transformation
It is not possible for a user to enumerate all the change
sequences that implement a transformation in a query
8

9. 9
initial state
sought-after state
1 (defn field-rename [esg]
2 (run* [?es]
3 (query-changes esg ?es [?orig-ast ?field]
4 (in-current-es [es ast]
5 (== ?orig-ast ast)
6 (ast-field ast ?field))
7 change->+
8 (in-current-es [es ast]
9 (fresh [?renamed ?new-name]
0 (ast-field ast ?renamed)
1 (ast-field|absent ast ?field)
2 (ast-field|absent ?orig-ast ?renamed)))))
public class Example {
int x = 0;
}
public class Example {
int y = 0;
}
Approach: Before&After AST Specification

10. 10
public class Example {
int x = 0;
int y = 1;
}
public class Example {
}
Source AST Target AST
1 3
2 4
Regular Dependency
List Dependency
Change Dependency Graph
1. insert(int x, Example, Example, nil, :BodyDeclarations, 0)
2. insert(int y, Example, Example, nil, :BodyDeclarations, 1)
3. insert(0, nil, int x, nil, :Initializer, nil)
4. insert(1, nil, int y, nil, :Initializer, nil)
Distilled Change Sequence
public class Example {
}
public class Example {
int x = 0;
int y = 1;
}
public class Example {
int x;
}
public class Example {
int x = 0;
}
public class Example {
int x = 0;
int y;
}
public class Example {
int x;
int y;
}
public class Example {
int x;
int y = 1;
}
public class Example {
int y;
}
public class Example {
int y = 1;
}
1
2
3
2
1
4
2
3 4
1
34
Evolution State Graph
Implementation Overview

11. 11
1 3
2 4
Regular Dependency
List Dependency
1. insert(int x, Example, Example, nil, :BodyDeclarations, 0)
2. insert(int y, Example, Example, nil, :BodyDeclarations, 1)
3. insert(0, nil, int x, nil, :Initializer, nil)
4. insert(1, nil, int y, nil, :Initializer, nil)
Solution: Change Dependency Graph
Insert Dependency List Dependency
Source SourceTarget Target
Move Dependency
Source Target
α

12. 12
Solution: Change Dependency Graph

13. 13
1. insert(int x, Example, Example, nil, :BodyDeclarations, 0)
2. insert(int y, Example, Example, nil, :BodyDeclarations, 1)
3. insert(0, nil, int x, nil, :Initializer, nil)
4. insert(1, nil, int y, nil, :Initializer, nil)
public class Example {
}
public class Example {
int x = 0;
int y = 1;
}
public class Example {
int x;
}
public class Example {
int x = 0;
}
public class Example {
int x = 0;
int y;
}
public class Example {
int x;
int y;
}
public class Example {
int x;
int y = 1;
}
public class Example {
int y;
}
public class Example {
int y = 1;
}
1
2
3
2
1
4
2
3 4
1
34
Solution: Evolution State Graph

14. • Detect instances of refactorings in open-source
projects using a single evolution query per
refactoring
• Ensure returned change sequences are minimal
and executable
• Compare our approach with the naive approach
K. Prete, N. Rachatasumrit, N. Sudan, and M. Kim,
“Template-based reconstruction of complex refactorings,”
in Proc. of the 2010 Int. Conf. on Software Maintenance (ICSM10)
E. Murphy-Hill, C. Parnin, and A. P. Black,
“How we refactor, and how we know it,”
Transactions on Software Engineering, vol. 38, pp. 5–18, 2012.
14
Evaluation: Outline

15. 15
ChangeNodes
Distiller
Known Refactorings
Magic
Constant
Field
Rename
Unused
Method
public class Example {
}
public class Example {
int x = 0;
int y = 1;
}
public class Example {
int x;
}
public class Example {
int x = 0;
}
public class Example {
int x = 0;
int y;
}
public class Example {
int x;
int y;
}
public class Example {
int x;
int y = 1;
}
public class Example {
int y;
}
public class Example {
int y = 1;
}
1
2
3
2
1
4
2
3 4
1
34
Evolution State Graph
Change Dependency Graph
1 3
2 4
Regular Dependency
List Dependency
Evolution Query
1 (query-changes esg ?es
2 [?absent ?method …]
3 (in-current-es [es ast]
4 (== ast ?absent)
5 (ast-method ast ?method)
6 (child+ ?method ?literal)
7 (literal-value ?literal ?value))
8 change->*
9 (in-current-es [es ast]
0 (ast-ast-field|introduced ?absent ast ?field)
1 (field-value|initialized ?field ?value)
2 (ast-method-method|corresponding …)
3 (child+ ?cmethod ?field-access)
4 (field-name|accessed ?field ?field-access)))
Minimal Executable Code Transformation
Insert Move Insert
Code Rev1
Code Rev2
Insert Move …
Evaluation: Outline

16. 16
1 1 public class Jar extends Zip {
2 2 private static final String INDEX_NAME="META-INF/INDEX.LIST";
3 + private static final String MANIFEST_NAME="META-INF/MANIFEST.MF";
3 4 private Manifest configuredManifest;
4 5 private Manifest savedConfiguredManifest;
5 6
6 7 protected void zipFile( ) throws IOException {
7 - if ("META-INF/MANIFEST.MF".equalsIgnoreCase(vPath)) {
8 + if (MANIFEST_NAME.equalsIgnoreCase(vPath)) {
8 9 if (!doubleFilePass || (doubleFilePass && skipWriting)) {
9 10 filesetManifest(fromArchive,is);
10 11 }
11 12 }
12 13 else {
13 14 super.zipFile(is,zOut,vPath,lastModified,fromArchive,mode);
14 15 }
15 16 }
16 17 }
Evaluation: Replace Magic Constant

17. 17
1 (query-changes esg ?es
2 [?absent ?method ?literal value ?cmethod ?field ?field-access]
3 (in-current-es [es ast]
4 (== ast ?absent)
5 (ast-method ast ?method)
6 (child+ ?method ?literal)
7 (literal-value ?literal ?value))
8 change->*
9 (in-current-es [es ast]
0 (ast-ast-field|introduced ?absent ast ?field)
1 (field-value|initialized ?field ?value)
2 (ast-method-method|corresponding ast ?method ?cmethod)
3 (child+ ?cmethod ?field-access)
4 (field-name|accessed ?field ?field-access)))
protected void zipFile(…) {
if (“META-INF/MANIFEST.MF”
.equalsIgnoreCase(vPath)) {
…
private String MANIFEST_NAME =
“META-INF/MANIFEST.MF";
…
protected void zipFile(…) {
if (MANIFEST_NAME
.equalsIgnoreCase(vPath)) {
…
Evaluation: Replace Magic Constant

18. 18
Legend
insert
delete
move
solution
dependency
update
• Total Changes: 74
• Solution Length Our Approach: 4
• Solution Length Replaying Sequence: 23
Evaluation: Inspecting the Solution

19. 19
Evaluation: Result Summary
NumberofChanges
0
50
100
150
200
250
300
350
Refactoring
Constant Constant Constant Constant Constant Constant Method Method Method Field Field Field Field
6
1513
3
39
12655
10
48
213
24
57
9
264
1214
118
199
1000
23
82
221
27
63
10
291
5
11
25
149
245
1244
74
202
Total Changes Solution Length Replaying Sequence Solution Length Our Approach

20. Conclusion
• The change equivalence problem renders specifying
a sought-after code transformation in terms of
changes difficult
• Our approach supports specifying code
transformations using before&after states in logic
queries
• Solutions to our queries are a minimal, executable
subsequence of changes that implements the sought-
after transformation
20

21. Discussion
Do ad-hoc solutions to the change equivalence
problem invalidate existing MSR studies?
21