• Like
  • Save
Applicative Logic Meta-Programming as the foundation for Template-based Program Transformation
Upcoming SlideShare
Loading in...5
×
 

Applicative Logic Meta-Programming as the foundation for Template-based Program Transformation

on

  • 319 views

Lecture about Ekeko and Ekeko/X at the Graduate School of Information Science and Technology at Osaka University.

Lecture about Ekeko and Ekeko/X at the Graduate School of Information Science and Technology at Osaka University.

Statistics

Views

Total Views
319
Views on SlideShare
246
Embed Views
73

Actions

Likes
0
Downloads
2
Comments
0

1 Embed 73

https://twitter.com 73

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Applicative Logic Meta-Programming as the foundation for Template-based Program Transformation Applicative Logic Meta-Programming as the foundation for Template-based Program Transformation Presentation Transcript

    • Applicative Logic Meta-Programming as the foundation for Template-based Program Transformation Coen De Roover
 cderoove@vub.ac.beSoftware Engineering Laboratory Software Languages Lab Osaka Brussels
    • Introducing Belgium: location and languages Dutch
 6.23 million French
 3.32 million German
 0.07 million Germany Luxemburg France Netherlands
    • Introducing Belgium: Brussels Jeanneke pis Zinneke pisAtomium Grote Markt - Grand Place Manneken pis
    • Introducing Belgium: painters Magritte (1964) Bruegel (1563) Rubens (1615) Ensor (1889) van Eyck (1434) van der Weyden (1445) Permeke (1929)
    • Introducing Belgium: food Waffles (from Brussels) Waffles (from Liège) Chocolates French Fries
    • Introducing Belgium: drinks For 25 little-known facts: http://cheeseweb.eu/2009/08/25-belgium/
    • Applicative Logic Meta-Programming as the foundation for Template-based Program Transformation Coen De Roover
 cderoove@vub.ac.beSoftware Engineering Laboratory Software Languages Lab Osaka Brussels
    • Motivation: repeating changes within a program annotation for properties of which evolution is to be tracked more typesafe version before after many public class BreakStatement extends Statement { @EntityProperty(value = SimpleName.class) private EntityIdentifier label; ! public EntityIdentifier getLabel() { return label; } ! public void setLabel(EntityIdentifier label) { this.label = label; } } public class BreakStatement extends Statement { @EntityProperty(value = SimpleName.class) private EntityIdentifier<SimpleName> label; ! public EntityIdentifier<SimpleName> getLabel() { return label; } ! public void setLabel(EntityIdentifier<SimpleName> label) { this.label = label; } }
    • Table II STUDY SUBJECTS Eclipse JDT core Eclipse SWT Mozilla Project Type IDE IDE Several projects associated with Internet Period of development 2001/06 ⇠ 2009/02 2001/05 ⇠ 2010/05 1998/03 ⇠ 2008/05 Study period 2004/07 ⇠ 2006/07 2004/07 ⇠ 2006/07 2003/04 ⇠ 2005/07 Total revisions 17000 revisions 21530 revisions 200000 revisions # of bugs 1812 1256 11254 # of Type I bugs 1405 (77.54%) 954 (75.96%) 7562 (67.19%) # of Type II bugs 407 (22.46%) 302 (24.04%) 3692 (32.81%) 2 3 4 5 6 7 Eclipse JDT core 0% 10% 20% 30% 40% 50% 60% 2 3 4 5 6 7 8 9 10 11 12 Percentage Eclipse SWT 0% 10% 20% 30% 40% 50% 60% 70% 2 4 6 8 10 12 14 16 18 21 25 29 Percentage Mozilla Figure 1. The number of times that the same bug is fixed Eclipse JDT core 0% 10% 20% 30% 40% 50% 60% 70% Percentage Eclipse SWT 0% 5% 10% 15% 20% 25% 30% 35% 0 1 3 4 7 1 6 1 8 6 8 4 1 5 9 3 4 7 4 Percentage Mozilla Table II STUDY SUBJECTS Eclipse JDT core Eclipse SWT Mozilla Project Type IDE IDE Several projects associated with Internet Period of development 2001/06 ⇠ 2009/02 2001/05 ⇠ 2010/05 1998/03 ⇠ 2008/05 Study period 2004/07 ⇠ 2006/07 2004/07 ⇠ 2006/07 2003/04 ⇠ 2005/07 Total revisions 17000 revisions 21530 revisions 200000 revisions # of bugs 1812 1256 11254 # of Type I bugs 1405 (77.54%) 954 (75.96%) 7562 (67.19%) # of Type II bugs 407 (22.46%) 302 (24.04%) 3692 (32.81%) 0% 10% 20% 30% 40% 50% 60% 70% 80% 2 3 4 5 6 7 Percentage Eclipse JDT core 0% 10% 20% 30% 40% 50% 60% 2 3 4 5 6 7 8 9 10 11 12 Percentage Eclipse SWT 0% 10% 20% 30% 40% 50% 60% 70% 2 4 6 8 10 12 14 16 18 21 25 29 Percentage Mozilla Figure 1. The number of times that the same bug is fixed 0% 10% 20% 30% 40% 50% 60% 70% 0 3 6 9 12 16 19 25 29 38 44 52 60 80 97 103 130 166 495 Percentage Eclipse JDT core 0% 10% 20% 30% 40% 50% 60% 70% 0 4 8 16 22 27 35 47 58 73 84 119 172 265 319 428 957 Percentage Eclipse SWT 0% 5% 10% 15% 20% 25% 30% 35% 0 31 63 94 127 161 196 231 278 336 398 464 531 585 659 733 804 887 1014 Percentage Mozilla Figure 2. The number of days taken for the supplementary fix to appear since an initial fix Table II STUDY SUBJECTS Eclipse JDT core Eclipse SWT Mozilla Project Type IDE IDE Several projects associated with Internet Period of development 2001/06 ⇠ 2009/02 2001/05 ⇠ 2010/05 1998/03 ⇠ 2008/05 Study period 2004/07 ⇠ 2006/07 2004/07 ⇠ 2006/07 2003/04 ⇠ 2005/07 Total revisions 17000 revisions 21530 revisions 200000 revisions # of bugs 1812 1256 11254 # of Type I bugs 1405 (77.54%) 954 (75.96%) 7562 (67.19%) # of Type II bugs 407 (22.46%) 302 (24.04%) 3692 (32.81%) 0% 10% 20% 30% 40% 50% 60% 70% 80% 2 3 4 5 6 7 Percentage Eclipse JDT core 0% 10% 20% 30% 40% 50% 60% 2 3 4 5 6 7 8 9 10 11 12 Percentage Eclipse SWT 0% 10% 20% 30% 40% 50% 60% 70% 2 4 6 8 10 12 14 16 18 21 25 29 Percentage Mozilla Figure 1. The number of times that the same bug is fixed 0% 10% 20% 30% 40% 50% 60% 70% 0 3 6 9 12 16 19 25 29 38 44 52 60 80 97 103 130 166 495 Percentage Eclipse JDT core 0% 10% 20% 30% 40% 50% 60% 70% 0 4 8 16 22 27 35 47 58 73 84 119 172 265 319 428 957 Percentage Eclipse SWT 0% 5% 10% 15% 20% 25% 30% 35% 0 31 63 94 127 161 196 231 278 336 398 464 531 585 659 733 804 887 1014 Percentage Mozilla Figure 2. The number of days taken for the supplementary fix to appear since an initial fix Motivation: repeating changes within a program correctly bugs requiring supplementary patches amount of supplements (~ missed occurrences) days before first supplement [“An empirical study of supplementary bug fixes” Park et al. MSR 2012]
    • Repeating Changes: using existing tool support requires specifying subjects of the transformation (LHS) their state afterwards or change actions (RHS) carefully ensuring no unwarranted changes are applied no required changes are missed where to apply a change what change to apply
    • Repeating Changes: using existing tool support Brussels’ logic meta-programming Ekeko Eclipse plugin applications program and corpus analysis program transformation meta-programming library for Clojure causally connected applicative meta-programming script queries over workspace specify code characteristics declaratively, leave search to logic engine manipulate workspace tool building Building Development Tools Interactively using the EKEKO Meta-Programming Library Coen De Roover Software Languages Lab Vrije Universiteit Brussel, Belgium Email: cderoove@vub.ac.be Reinout Stevens Software Languages Lab Vrije Universiteit Brussel, Belgium Email: resteven@vub.ac.be Abstract—EKEKO is a Clojure library for applicative logic meta-programming against an Eclipse workspace. EKEKO has been applied successfully to answering program queries (e.g., “does this bug pattern occur in my code?”), to analyzing project corpora (e.g., “how often does this API usage pattern occur in this corpus?”), and to transforming programs (e.g., “change occurrences of this pattern as follows”) in a declarative manner. These applications rely on a seamless embedding of logic queries in applicative expressions. While the former identify source code of interest, the latter associate error markers with, compute statistics about, or rewrite the identified source code snippets. In this paper, we detail the logic and applicative aspects of the EKEKO library. We also highlight key choices in their implemen- tation. In particular, we demonstrate how a causal connection with the Eclipse infrastructure enables building development tools interactively on the Clojure read-eval-print loop. I. INTRODUCTION EKEKO is a Clojure library that enables querying and manipulating an Eclipse workspace using logic queries that are seamlessly embedded in functional expressions. Recent applications of EKEKO include the GASR tool for detect- ing suspicious aspect-oriented code [1] and the QWALKEKO tool for reasoning about fine-grained evolutions of versioned code [2]. In this paper, we describe the meta-programming facilities offered by EKEKO and highlight key choices in their implementation1 . We also draw attention to the highly interactive manner of tool building these facilities enable. II. RUNNING EXAMPLE: AN ECLIPSE PLUGIN More concretely, we will demonstrate how to build a lightweight Eclipse plugin entirely on the Clojure read-eval- print loop. We will use this plugin as a running example throughout the rest of this paper. Our Eclipse plugin is to support developers in repeating similar changes throughout an entire class hierarchy. It is to associate problem markers with fields that have not yet been changed. In addition, it is to present developers a visualization of these problems. Finally, it is to provide a “quick fix” that applies the required changes correctly. Figure 1 illustrates the particular changes that need to be repeated. The raw EntityIdentifier type of those fields within a subclass of be.ac.chaq.model.ast.java.ASTNode 1The EKEKO library, its implementation, and all documentation is freely available from https://github.com/cderoove/damp.ekeko/. public c l a s s BreakStatement extends Statement { / / Before changes : @EntityProperty ( value = SimpleName . c l a s s ) private E n t i t y I d e n t i f i e r l a b e l ; / / A f t e r changes : @EntityProperty ( value = SimpleName . c l a s s ) private E n t i t y I d e n t i f i e r <SimpleName> l a b e l ; / / . . . ( a . o . , accessor methods change a c c o r d i n g l y ) } Fig. 1: Example changes to be repeated. that carry an @EntityProperty annotation, is to receive a type parameter that corresponds to the annotation’s value key. III. ARCHITECTURAL OVERVIEW The EKEKO library operates upon a central repository of project models. These models contain structural and behavioral information that is not readily available from the projects themselves. The models for Java projects include abstract syntax trees provided by the Eclipse JDT parser, but also control flow and data flow information computed by the SOOT program analysis framework [3]. An accompanying Eclipse plugin automatically maintains the EKEKO model repository. To this end, it subscribes to each workspace change and triggers incremental updates or complete rebuilds of project models. As a result, the infor- mation operated upon by the EKEKO library is always up-to- date. In addition, this plugin provides an extension point that enables registering additional kinds of project models. The KEKO extension, for instance, builds its project model from the results of a partial program analysis [4] —enabling queries over compilation units that do not build correctly. IV. LOGIC PROGRAM QUERYING The EKEKO library enables querying and manipulating pro- grams using logic queries and applicative expressions respec- tively. We detail the former first. Section V discusses the latter. The program querying facilities relieve tool builders from implementing an imperative search for source code that ex- hibits particular characteristics. Instead, developers can specify these characteristics declaratively through a logic query. The [WCRE-CSMR14]
    • Think of it asquerying a databaseof programinformation! Logic meta-programming (LMP)
    • Logic relations: ast/2 and has/3 “tables” relation between a ?node
 of an Abstract Syntax Tree (AST)
 and its ?type (ast  ?type  ?node) (has  ?property  ?node  ?value) LM P relation of AST nodes and the values of their properties
    • (ekeko  [?statement  ?expression]      (ast  :ReturnStatement  ?statement)        (has  :expression  ?statement  ?expression))   Logic querying: AST relations SELECT  ast.node  as  ?statement,                  has.value  as  ?expression   FROM      ast,  has     WHERE    ast.type  =    :ReturnStatement  AND                has.property  =  :expression  AND                has.node  =  ast.node;                 SQL query relation of all AST nodes of type :ReturnStatement and the value of their :expression property equivalent logic query LM P
    • (defn        expression|returned      [?expression]      (fresh  [?statement]          (ast  :ReturnStatement  ?statement)            (has  :expression  ?statement  ?expression))) Logic programming: defining relations LM P local variable (ekeko*  [?returned  ?type]      (expression|returned  ?returned)      (ast|expression-­‐type  ?returned  ?type)      (type|binary  ?type) returned expressions of a type defined in compiled rather than source code defining relation: using newly defined relation:
    • control and data flow
 structural syntactic for(Object i : collection) Ekeko Library: relations class Ouch { int hashCode() { return ...; } } scanner = new Scanner(); ... x.close(); ... scanner.next(); (ast ?kind ?ast) (has ?property ?ast ?value) (ast-encompassing|method+ ?ast ?m) (ast-encompassing|type+ ?ast ?t) (classfile-type ?binaryfile ?type) (type-type|sub+ ?type ?subtype) (type-name|qualified ?type ?qname) (advice-shadow ?advice ?shadow) for logic meta-programming (method|soot-cfg ?m ?cfg) (unit|soot-usebox ?u ?ub) (local|soot-pointstoset ?l ?p) (soot|may|alias ?l1 ?l2)
    • Ekeko Library: functions (remove-node node) (replace-node node newnode) (change-property node property value) (apply-and-reset-rewrites!) for applicative meta-programming (visualize nodes edges :layout layout :node|label labelfn :edge|label labelfn . . .) (add-problem-marker marker node) (register-quickfix marker rewritefn) (reduce—workspace fn initval) (wait-for-builds-to-finish) rewriting visualizing tooling
    • before after public class BreakStatement extends Statement { @EntityProperty(value = SimpleName.class) private EntityIdentifier label; ! public EntityIdentifier getLabel() { return label; } ! public void setLabel(EntityIdentifier label) { this.label = label; } } public class BreakStatement extends Statement { @EntityProperty(value = SimpleName.class) private EntityIdentifier<SimpleName> label; ! public EntityIdentifier<SimpleName> getLabel() { return label; } ! public void setLabel(EntityIdentifier<SimpleName> label) { this.label = label; } } Repeating Changes: using Ekeko Live Dem o!
    • Repeating Changes: using Ekeko specifying changes requires significant expertise and most state of the art tools often requires multiple iterations Imperative Program Transformation by Rewriting 53 says that an assignment x := v (where v is a variable) can be replaced by = c if the “last assignment” to v was v := c (where c is a constant). The side ndition formalises the notion of “last assignment”, and will be explain later in e paper: n : (x := v) =⇒ x := c if n ⊢ A△ (¬def(v) U def(v) ∧ stmt(v := c)) conlit(c) The rewrite language has several important properties: The specification is in the form of a rewrite system with the advantages of succinctness and intuitiveness mentioned above. The rewrite system works over a control flow graph representation of the program. It does this by identifying and manipulating graph blocks which are based on the idea of basic blocks but with finer granularity. The rewrites are executable. An implementation exists to automatically de- termine when the rewrite applies and to perform the transformation just from the specification. no unwarranted changes are applied no required changes are missed generalization refinement
    • Repeating Changes : introducing Ekeko/X more intuitively 1/ template-based transformation specifications 2/ matching and rewriting directives in templates 3/ template mutation operators Towards
    • 1/ template-based transformation specifications 2/ matching and rewriting directives in templates 3/ template mutation operators (ekeko/x   !      ?exp.setAge(?arg)   !      =>   !      new  Integer(?arg))   (ekeko/x <LHS1>…<LHSn> => <RHS1>…<RHSn>) template on LHS: matches identify subjects template on RHS: generates code for every LHS match
    • 1/ template-based transformation specifications 2/ matching and rewriting directives in templates 3/ template mutation operators (ekeko/x        ?exp.setAge(?arg)        =>        new  Integer(?arg))   will not match setAge(5) no destination for generated code (ekeko/x        [?exp]@[relax-­‐receiver].setAge(?arg)        =>        [new  Integer(?arg)]@[(replace  ?arg)])   [<component>]@[<directive>] matching directive rewriting directive
    • 1/ template-based transformation specifications 2/ matching and rewriting directives in templates 3/ template mutation operators []@[orlarger] class ?name extends [Component]@[orsubtype] { [[public]@[member]  void acceptVisitor(?type ?param) {[ System.out.println(?string); [?x]@[(must-alias ?param)].?visitMethod(this); ]@[orflow] }]@[member] } public class OnlyLoggingLeaf extends Component { public void acceptVisitor(ComponentVisitor v) { System.out.println("Only logging."); } } public class SillyLeaf extends OnlyLoggingLeaf { public void acceptVisitor(ComponentVisitor v) { super.acceptVisitor(v); ComponentVisitor temp = v; temp.visitSuperLogLeaf(this); } } rich inter-proceduralsemantics!
    • 1/ template-based transformation specifications 2/ matching and rewriting directives in templates 3/ template mutation operators []@[orlarger] class ?name extends [Component]@[orsubtype] { [[public]@[member]  void acceptVisitor(?type ?param) {[ System.out.println(?string); [?receiver]@[(must-alias ?param)].?visitMethod(this); ]@[orflow] }]@[member] } public class OnlyLoggingLeaf extends Component { public void acceptVisitor(ComponentVisitor v) { System.out.println("Only logging."); } } public class SillyLeaf extends OnlyLoggingLeaf { public void acceptVisitor(ComponentVisitor v) { super.acceptVisitor(v); ComponentVisitor temp = v; temp.visitSuperLogLeaf(this); } } rich inter-proceduralsemantics!
    • 1/ template-based transformation specifications 2/ matching and rewriting directives in templates 3/ template mutation operators return  age;   return  ?exp;introduce-variable generalize-aliases atomic generalization compound generalization public  boolean  hasChildren(Object  element)  {     if  (element  ==  null)       return  false;     return  getChildren(element).length  >  0;   } public  boolean  hasChildren(Object  ?param)  {     if  ([?ref1]@[(must-­‐alias  ?param)]  ==  null)       return  false;     return  getChildren([?ref2]@[(must-­‐alias  ?param)]).length  >  0;   }
    • 1/ template-based transformation specifications 2/ matching and rewriting directives in templates 3/ template mutation operators return  age;   return  ?exp;introduce-variable generalize-aliases atomic generalization compound generalization public  boolean  hasChildren(Object  element)  {     if  (element  ==  null)       return  false;     return  getChildren(element).length  >  0;   } public  boolean  hasChildren(Object  ?param)  {     if  ([?ref1]@[(must-­‐alias  ?param)]  ==  null)       return  false;     return  getChildren([?ref2]@[(must-­‐alias  ?param)]).length  >  0;   }
    • Live Dem o!
    • Applications: beyond repeating changes 1 List<User> getRoleUser () { 2 List<User> listUsers = new ArrayList<User>(); 3 List<User> users = this.userDao.getUsers(); 4 List<Role> roles = this.roleDao.getRoles(); 5 for (User u : users) { 6 for (Roles r : roles) { 7 if (u.roleId().equals(r.roleId())) { 8 User userok = u; 9 listUsers.add(userok); 10 }}} 11 return listUsers; 12 } Figure 1: Sample code that implements join operation in applica- tion code, abridged from actual source for clarity 1 List listUsers := [ ]; int i, j = 0; 2 List users := Query(SELECT ⇤ FROM users); 3 List roles = Query(SELECT ⇤ FROM roles); 4 while (i < users.size()) { 5 while (j < roles.size()) { 6 if (users[i].roleId = roles[j].roleId) 7 listUsers := append(listUsers, users[i]); 8 ++j; 9 } 10 ++i;} Figure 2: Sample code expressed in kernel language c 2 constant ::= True | False | number literal | string litera e 2 expression ::= c | [ ] | var | e.f | {fi = ei } | e1 op e2 | ¬ | Query(...) | size(e) | geter (es ) | append(er , es ) | unique(e) c 2 command ::= skip | var := e | if(e) then c1 else c2 | while(e) do c | c1 ; c2 | assert e op 2 binary op ::= ^ | _ | > | = Figure 4: Abstract syntax of the kernel language Entry Point Identifier Java Inferred SQL Java Code Fragment Identifier Java Code fragment Java Application source + config files Code Inliner Inlined persistent data methods VC Computation Invariant + Postcondition Synthesizer Formal Verification Transform method b Java Java XML Kernel Language Compiler Figure 5: QBS architecture 9 listUsers.add(userok); 10 }}} 11 return listUsers; 12 } Figure 1: Sample code that implements join operation in applica- tion code, abridged from actual source for clarity 1 List listUsers := [ ]; int i, j = 0; 2 List users := Query(SELECT ⇤ FROM users); 3 List roles = Query(SELECT ⇤ FROM roles); 4 while (i < users.size()) { 5 while (j < roles.size()) { 6 if (users[i].roleId = roles[j].roleId) 7 listUsers := append(listUsers, users[i]); 8 ++j; 9 } 10 ++i;} Figure 2: Sample code expressed in kernel language Postcondition listUsers = ⇡`(./' (users, roles)) where '(eusers , eroles ) := eusers .roleId = eroles .roleId ` contains all the fields from the User class Translated code 1 List<User> getRoleUser () { 2 List<User> listUsers = db.executeQuery( 3 "SELECT u 4 FROM users u, roles r 5 WHERE u.roleId == r.roleId 6 ORDER BY u.roleId, r.roleId"); 7 return listUsers; } Figure 3: Postcondition as inferred from Fig. 1 and code after query transformation domain-specific optimizations API migration program renovation domain-specific refactorings refactorings to eliminate or maintain clones convert anonymous classes to Java 8 lambda expressions migrate an application from using SWING to using SWT [Cheung et al., PLDI13]
    • Ongoing work involving Ekeko/X and possible collaborations summarizing similar code fragments into template action: template mutations goal: template that covers all fragments start state: code fragment state space search efficacy of atomic vs compound mutation operators
    • Ongoing work involving Ekeko/X and possible collaborations buggy fixed evolved clone B clone B evolved clone B fixed system variant A system variant B system variant C evolved clone C clone C evolved clone C fixed PATCH PATCH' PATCH''?? supporting clone-and-own reuse repeat changes across variants impact analysis of divergences on transformation by mutating transformation
    • Conclusion