Building DSLs with the Spoofax Language Workbench

1,890 views

Published on

Slides for my keynote talk at Code Generation 2010

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,890
On SlideShare
0
From Embeds
0
Number of Embeds
109
Actions
Shares
0
Downloads
65
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Building DSLs with the Spoofax Language Workbench

  1. 1. Building DSLs with The Spoofax Language Workbench Eelco Visser Delft University of Technology http://eelcovisser.org Joint work with Lennart Kats, Karl-Trygve Kalleberg, Maartje de Jonge, and many others
  2. 2. The Meta-Languages Wars Internal DSLs vs External DSLs Textual vs Graphical Partial vs Complete Code Generation Project-Specific vs Multi-Project DSLs Code Generation vs Interpretation Dynamically vs Statically Typed
  3. 3. The Meta-Languages Wars Internal DSLs vs External DSLs Textual vs Graphical Partial vs Complete Code Generation Project-Specific vs Multi-Project DSLs Code Generation vs Interpretation Dynamically vs Statically Typed
  4. 4. WebDSL http://webdsl.org separation of concerns & linguistic integration Eelco Visser. WebDSL: A Case Study in Domain-Specific Language Engineering. GTTSE 2007: 291-373
  5. 5. 1,190,303 publications http://researchr.org
  6. 6. publication records correct & extend
  7. 7. author profiles
  8. 8. bibliographies tagging reputation system access control rules user groups conference calendar community engineering etc.
  9. 9. 18,000 lines of WebDSL code 138 (generated) tables in mysql
  10. 10. Concerns in web applications - Data model - User interface templates - Actions - Access control rules - Data validation WebDSL - Linguistic integration - Cross concern, static consistency checking
  11. 11. WebLib: a model of researchr
  12. 12. Data Model entity Publication { key :: String (id) title :: String (name,searchable) authors -> List<Author> journal :: String (searchable) volume :: Int issue :: Int year :: Int (searchable) month :: String (searchable) firstpage :: Int lastpage :: Int cites -> Set<Publication> citedBy -> Set<Publication> (inverse=Publication.cites) } entity Author { key :: String (id) name :: String publications -> Set<Publication> (inverse=Publication.authors) created :: DateTime modified :: DateTime }
  13. 13. User Interface
  14. 14. Full Text Search define page search() { var query : String action show() { var pubs : List<Publication> := searchPublication(query); if(pubs.length > 0) { replace(results, results(query,pubs)); } else { replace(results, noresults()); entity Publication { title :: String (name,searchable) } journal :: String (searchable) } year :: Int (searchable) main{ month :: String (searchable) section{ ... } header{"Search"} form{ input(query)[onkeyup:=show()] } placeholder results{} } } }
  15. 15. Page & Template Definitions define page root() { main{ section{ header{"Welcome to WebLib: Your Digital Library"} aboutWeblib() recent() tips() } } } define aboutWeblib() { par{ "WebLib is a digital library ..." } }
  16. 16. XML Embedding define no-span main() { <div class="page-wrap"> top() <div class="body"> messages() elements() </div> <div class="clear"></div> </div> footer() }
  17. 17. Page Navigation define tips() { section{ header{"What Can You Do Here?"} list{ listitem{ navigate(newauthor()){ "Add an author" } } listitem{ navigate(newpublication()){ "Add a publication" } } listitem{ navigate(search()){ "Search" } } } } } define page search() { ... }
  18. 18. HQL Embedding extend entity Person { publications -> List<Publication> := select p from Publication as p, Author as a where (a.person = ~this) and (p = a.publication) }
  19. 19. Forms & Data Binding define page editpublication(pub : Publication) { main{ section{ header{"Edit Publication '" output(pub.title) "'"} form{ group{ label("Key:" ){ input(pub.key) } label("Title:" ){ input(pub.title) } label("Authors:"){ input(pub.authors) } ... } submit action{ pub.modified := now(); return publication(pub); } { "Save" } } } } }
  20. 20. Access Control Danny M. Groenewegen, Eelco Visser. Declarative Access Control for WebDSL: Combining Language Integration and Separation of Concerns. ICWE 2008: 175-188
  21. 21. Access Control Rules define showReview(review : Review) { action delete() { ... } section{ ... navigate(editReview(review)){"[Edit]"} submitlink delete() {"[X]"} } }
  22. 22. Access Control Rules define showReview(review : Review) { action delete() { ... } section{ ... navigate(editReview(review)){"[Edit]"} submitlink delete() {"[X]"} } } access control rules principal is User with credentials username, password predicate isReviewer(review : Review) { review.reviewer == securityContext.principal } rule template showReview(review : Review, pubLink : Bool) { review.public || isReviewer(review) rule action delete() { loggedIn() && isReviewer(review) } } rule page editReview(review : Review) { loggedIn() && isReviewer(review) }
  23. 23. Data Validation Danny M. Groenewegen, Eelco Visser. Integration of Data Validation and User Interface Concerns in a DSL for Web Applications. SLE 2009: 164-173
  24. 24. Data Validation: Form Input Validation label("Acronym (without year)"){ input(acronym){ validate(!isEmptyString(acronym), "Provide acronym") validate(!(/[ ]/.find(acronym)), "Acronym should not contain spaces") validate(!(/[0-9][0-9][0-9][0-9]/.find(acronym)), "Acronym should not contain year") } }
  25. 25. Data Validation: Data Invariants entity Publication { key :: String (id, index(25), validate(isUniquePublication(this), "that key is already in use"), validate(isValidKeyForURL(key), "Key should consist of letters, digits, : and -")) ... }
  26. 26. Data Validation: Assertions extend entity Person { invitation -> Invitation (inverse=Invitation.invitee) function invite(email : Email, invitation : WikiText) : Invitation { validate(findUser(email) == null && findInvitationByEmail(email).length == 0, "That email address is already in use."); ... } }
  27. 27. Concerns in web applications - Data model - User interface templates - Actions - Access control rules - Data validation WebDSL - Linguistic integration - Cross concern, static consistency checking
  28. 28. IDEs for DSLs
  29. 29. parse check Model AST Errors desugar Traditional Compiler core Architecture generate Code
  30. 30. Compiler Ingredients Syntax definition ★ concrete syntax ★ abstract syntax Static semantics ★ error checking ★ name resolution ★ type analysis Model-to-model transformation ★ express constructs in core language Code generation ★ translate core language models to implementation
  31. 31. IDEs Required
  32. 32. Editor Services for Modern IDEs Syntactic Editor Services ★ syntax checking ★ bracket matching ★ syntax highlighting ★ code folding ★ outline view Semantic Editor Services ★ error checking ★ reference resolving ★ hover help ★ code completion ★ refactoring
  33. 33. parse Editor AST feedback core Integrated Development Environment Code
  34. 34. Architectural Requirements for IDEs Editor services are based on AST ★ services analyze structure, not text of source Error recovery ★ continue services in presence of syntactic errors Incremental processing ★ effort of applying analysis, transformation, generation should be proportional to change in source Separate compilation (analysis, transformation) ★ keep track of dependencies
  35. 35. Holy Grail of Software Language Definition Automatically derive efficient, scalable, incremental compiler + usable IDE from high-level, declarative language definition
  36. 36. The Spoofax Language Workbench
  37. 37. The Spoofax/IMP Language Workbench Syntax definition: SDF ★ declarative, modular syntax definition Transformation, analysis, generation: Stratego ★ rewrite rules, strategies, dynamic rules Editor services ★ domain-specific languages for configuration Based on IMP framework by IBM Research
  38. 38. Syntax Definition
  39. 39. Syntax Definition Pretty-Print Parse Table Signature Table Parse Transform Pretty-Print @Entity entity User { class User { name :: String String _user; pw :: Secret public User } getUser() { def output(u : syntax definition is basis of language definition return _user; User) { }
  40. 40. Lexical & Context-Free Syntax
  41. 41. Structure of Sentences composed from
  42. 42. Context-Free Grammar sort regular expression symbol Definition grammar production Property* {Stat “;”}* “entity”
  43. 43. Lexical Syntax regular expression character class characters in lexical syntax not separated by layout
  44. 44. Syntax of Data Models
  45. 45. Recognizing Well-Formed Sentences
  46. 46. Lexical Syntax: Identifiers & Literals follow restriction: symbol may not reserved words be followed by character from this class: BlogEntry is one ID
  47. 47. Lexical Syntax: Strings escape complement escape escape
  48. 48. Lexical Syntax: Whitespace LAYOUT? is inserted between symbols in CF productions normalize
  49. 49. Modular Syntax Definition
  50. 50. Parsing & Abstract Syntax building structured representation of sentence
  51. 51. Linking Concrete & Abstract Syntax
  52. 52. Terms to Represent Abstract Syntax Entity(“Blog”, [Property(“name”,SimpleType(“String”)), Property(“author”, SimpleType(“User”))] )
  53. 53. Parsing in Spoofax
  54. 54. Signatures defining the structure of abstract syntax terms
  55. 55. Syntax Definition Pretty-Print Parse Table Signature Table Parse Transform Pretty-Print @Entity entity User { class User { name :: String String _user; pw :: Secret public User } getUser() { def output(u : syntax definition is basis of language definition return _user; User) { }
  56. 56. Signatures declare arguments and types of constructors signature is generated from grammar
  57. 57. Disambiguation
  58. 58. Arithmetic Expressions note: this style produces good abstract syntax Plus(Times(Var(“x”),Int(2)),Var(“z”))
  59. 59. Associativity
  60. 60. Priority
  61. 61. Parsing after Disambiguation
  62. 62. Brackets
  63. 63. Language Composition
  64. 64. Embedding HQL module expressions imports Common imports HQL exports context-free syntax SelectStatement -> Exp define page index() { list{ for(p : Post in select p from Post as p order by (p.title)) { listitem{ output(p) } } } } ForElem("p", SimpleType("Post") , QueryRule( SelectFrom( Some(Select(None(), [AliasedExpression(Path(["p"]), None())])) , FromClause([FromRangeJoin(FromClass(Path(["Post"]), Some(AsAlias(Alias("p" ) , Some(OrderByClause([OrderElement(Paren([Path(["p", "title"])]), None())])) ) , [CallElems("listitem", [CallArgs("output", [Var("p")])])] )
  65. 65. Presentational Editor Services
  66. 66. Editor Services Syntax highlighting ★ token coloring Code folding ★ folding & unfolding code fragments Outline view ★ table of contents Syntax properties ★ bracket matching, indentation Syntax completion ★ match on syntactic triggers & replace with template
  67. 67. Editor Services Composition module nwl.main imports nwl-Builders nwl-Colorer nwl-Folding nwl-Outliner nwl-References nwl-Syntax nwl-Completions language General properties name : nwl id : nwl extends : Root description : "Spoofax/IMP-generated editor for the nwl language" url : http://strategoxt.org extensions : nwl table : include/nwl.tbl start symbols : Start
  68. 68. Syntax Definition Default Syntax Default Default Highlighting Code Folding Outline View Custom Syntax Custom Custom + Highlighting + Code Folding + Outline View Editor
  69. 69. Code Folding module nwl-Folding.generated folding Default folding Start.Module Definition.Entity Property.Property Definition.TemplateDef Element.CallArgs PageRef.PageRef Element.Action module nwl-Folding imports nwl-Folding.generated folding Element.CallElems Element.Call Element.ForElem Element.ForAllElem
  70. 70. Outline View module nwl-Outliner.generated outliner Default outliner Start.Module Definition.Entity Property.Property Definition.TemplateDef Element.CallArgs Exp.MethodCall Element.CallElems Element.Call Element.ForElem Element.ForAllElem PageRef.PageRef Element.Action Statement.For Statement.While Statement.If Element.Submit Element.XmlElem
  71. 71. module nwl-Colorer.generated colorer Default, token-based highlighting Default Syntax Highlighting keyword : 127 0 85 bold identifier : default string : blue number : darkgreen var : 255 0 100 italic operator : 0 0 128 layout : 100 100 0 italic colorer System colors darkred = 128 0 0 red = 255 0 0 darkgreen = 0 128 0 green = 0 255 0 darkblue = 0 0 128 blue = 0 0 255 cyan = 0 255 255 magenta = 255 0 255 yellow = 255 255 0 white = 255 255 255 black = 0 0 0 gray = 128 128 128 grey = gray orange = 255 165 0 pink = 255 105 180 brown = 139 69 19 default = _
  72. 72. Custom Syntax Highlighting module nwl-Colorer imports nwl-Colorer.generated colorer Element : darkgreen
  73. 73. Syntax Properties module nwl-Syntax.generated language Syntax properties (static defaults) // Comment constructs: line comment : "//" block comment : "/*" * "*/" // Fences (used for matching, // inserting, indenting brackets): fences : [ ] ( ) { } // Automatic indent hints // (indent after these tokens): indent after : "=" ":" // Regular expression for identifiers: identifier lexical : "[A-Za-z0-9_]+"
  74. 74. Parse Error Recovery
  75. 75. Parse Error Recovery also for composite languages
  76. 76. Builders Semantic Editor Services
  77. 77. Builders: Analysis & Transformation Services module nwl-Builders builders provider: include/nwl.ctree observer: editor-analyze builder : "Generate Java code" = generate-java (openeditor) (realtime) builder : "Show ATerm (selection)" = generate-aterm (openeditor) (realtime) (meta) builder : "Normalize (selection)" = show-normalized (openeditor) (realtime) (meta) builder : "Normalize Pretty (selection)" = show-normalized-pp (openeditor) (realtime) (meta)
  78. 78. Term Rewriting
  79. 79. Term Rewriting Term rewrite rules ★ transform term to term ★ pattern matching ★ variable binding ★ substitution Rewriting strategy ★ algorithm for applying rewrite rules
  80. 80. Term Rewrite Rule left-hand side pattern label/name desugar : Property(x, t) -> Property(x, t, []) variable right-hand side pattern
  81. 81. Rewrite Strategy generic strategy strategy definition desugar-all = innermost(desugar) strategy instantiation apply transformation s exhaustively to all sub- innermost(s) terms of subject term in bottom-up order
  82. 82. Constant Folding y := a + (3 * 5 + 2); Assign( Var("y") , Plus( Var("a") , Plus(Times(IntLit("3"), IntLit("5")), IntLit("2")) parse ) ) Assign( Var("y") , BinOp(Var("a"), "+", IntLit("17")) desugar + eval ) y := (a + 17); pretty-print
  83. 83. Constant Folding Rules strategies eval-all = innermost(desugar + eval) rules eval : BinOp(IntLit(x), "+", IntLit(y)) -> IntLit(<addS>(x, y)) eval : BinOp(IntLit(x), "-", IntLit(y)) -> IntLit(<subtS>(x, y)) eval : BinOp(IntLit(x), "*", IntLit(y)) -> IntLit(<mulS>(x, y)) eval : BinOp(IntLit(x), "/", IntLit(y)) -> IntLit(<divS>(x, y))
  84. 84. Conditional Rewrite Rules bound in condition eval : BinOp(IntLit(x), "+", IntLit(y)) -> IntLit(z) condition where z := <addS>(x, y) match apply transformation eval : BinOp(IntLit(x), "+", IntLit(y)) -> IntLit(<addS>(x, y))
  85. 85. Code Generation by Model Transformation
  86. 86. Rewriting with Concrete Object Syntax elem-to-java-servlet : tc@elem|[ x(e)[passign*]{elem*} ]| -> <wrap-input-render-java-code> bstm* |[ String x_temp = ident+"~inputident"+uniqueid; ~*<render-error-messages-with-error-template(|java:expr|[ x_temp ]|, <ErrorTemplateInput>)> bstm*|[ bstm_call* ]| ]| where <is-input-template> tc with inputident := <get-inputnumber> tc ; x_temp := <newname> "temp" ; bstm_call* := <control-flow-tcall-helper(|"render",expr|[ x_temp ]|)> tc Stratego + embedded Java
  87. 87. Consistency Checking
  88. 88. Consistency Checking Syntax definition ★ what are well-formed sentences? Static analysis ★ not all ‘well-formedness’ properties are context-free ★ consistency of compositions ★ consistency of expressions wrt declarations Error reporting ★ indicate errors in editor ★ use sensible error message
  89. 89. Consistency Checking: Ingredients Editor Interface ★ collecting and displaying errors, warnings Error checking ★ checking static constraints and reporting errors Type analysis ★ computing types of expressions Name resolution ★ disambiguation of names Reference resolving ★ linking identifiers to declarations
  90. 90. Consistency Checking: Generic Approach Rename ★ make identifiers unique Map ★ map identifiers to declarations Project ★ compute properties of declarations, expressions Check ★ check constraints
  91. 91. Editor Interface module nwl-Builders imports nwl-Builders.generated builders provider: include/nwl.ctree observer: editor-analyze editor/nwl-Builders.esv trans/static-analysis.str module static-analysis imports include/nwl imports entities imports utils rules // static analysis editor-analyze: (ast, path, fullpath) -> (errors, warnings, notes) with ...
  92. 92. Editor Interface editor-analyze: (ast, path, fullpath) -> (errors, warnings, notes) with errors := <collect-all(check, conc)> ast; warnings := <collect-all(constraint-warning, conc)> ast; notes := <collect-all(constraint-note, conc)> ast
  93. 93. Error Checking Rules check : context -> (target, error) where assumption where require(constraint) require(s) = not(s) – Context: identifying points in the code to check – Assumptions: only report an error if certain assumptions hold (validating the context and avoiding spurious errors) – Constraints: checking for constraints at the context – Formulating an error message – Attribution of the error to a particular character range in the source text (usually, only part of the context
  94. 94. Error Checking: Binary Operators check : e@BinOp(e1, op, e2) -> (e, $[operator [op] not defined for [<pp>t1] and [<pp>t2]]) where t1 := <type-of> e1 where t2 := <type-of> e2 where require(<type-of> e)
  95. 95. Pretty-Printing with String Interpolation pp : Entity(x, prop*) -> $[entity [x] { [<map(pp)>prop*] }] pp : Property(x,t) -> $[[x] : [<pp>t] ] pp : SimpleType(x) -> x pp : SetType(t) -> $[Set<[<pp> t]>] pp : [] -> $[] pp : [t] -> <pp>t pp : [t1,t2|ts] -> $[[<pp>t1],[<pp>[t2|ts]]]
  96. 96. Type Analysis type-of : e -> t compute type of expression
  97. 97. Type Analysis: Literals type-of : StringLit(x) -> SimpleType("String") type-of : IntLit(x) -> SimpleType("Int")
  98. 98. Type Analysis: Binary Operators type-of : BinOp(e1, op, e2) -> t where t := <function-type>(op, [<type-of>e1, <type-of>e2]) function-type : ("+", [SimpleType("String"), SimpleType("String")]) -> SimpleType("String") function-type : ("+", [SimpleType("Int"), SimpleType("Int")]) -> SimpleType("Int") function-type : ("-", [SimpleType("Int"), SimpleType("Int")]) -> SimpleType("Int")
  99. 99. Type Analysis: What is Type of Variable? define page root(x : Int) { action exptest() { for(y : Int in {1,2,x}) { x := x + y; } } type-of : } Var(x) -> t where t := ??? Assign( Var("x") , BinOp(Var("x"), "+", Var("y")) ) type of variable not part of variable use
  100. 100. Variables: Map declare-all = alltd(declare) declare : Param(x, t) -> Param(x, t) with rules( TypeOf : x -> t ) type-of : Var(x) -> t where t := <TypeOf> x
  101. 101. Scope define page root(x : Int) { action exptest() { for(x : Int in {1,2,x}) { print(x); } } } multiple occurrences of same identifier corresponding to different declarations
  102. 102. Variables: Map + Rename rename-all = alltd(rename) rename : Param(x, t) -> Param(y, t) with y := <rename-var>(x, t) unique annotation rename-var : (x, t) -> y with y := x{<new>}; map variable to type rules( TypeOf : y -> t RenameId : x -> y ) rename occurrences rename : Var(x) -> Var(y) where y := <RenameId> x type-of : Var(x) -> t where t := <TypeOf> x
  103. 103. Term Annotations t{t1,...,tn} add additional information to term without affecting signature
  104. 104. Variables: Check check : e@Var(x) -> (e, $[Variable '[x]' not declared]) where require(<type-of>e)
  105. 105. Variable Binding Constructs rename : For(x, t, e1, stat1*) -> For(y, t, e2, stat2*) with e2 := <rename-all> e1 with {| RenameId : y := <rename-var>(x, t) ; stat2* := <rename-all> stat1* |} For defines local variable x in body stat1*’
  106. 106. Editor Interface with Analysis editor-analyze: (ast, path, fullpath) -> (errors, warnings, notes) with ast2 := <analyze> ast; errors := <collect-all(check, conc)> ast2; warnings := <collect-all(constraint-warning, conc)> ast2; notes := <collect-all(constraint-note, conc)> ast2 analyze = rename-all
  107. 107. Rename, Map, Project, Check Rename ★ make local variables unique Map ★ variables to their type Project ★ compute type of expressions Check ★ check constraints using types
  108. 108. Reference Resolution
  109. 109. Reference Resolution Aiding program navigation ★ Hover-click on identifier to jump to declaration Reuse name resolution infrastructure
  110. 110. Reference Resolution module nwl-References imports nwl-References.generated references reference _ : editor-resolve editor-resolve: (source, position, ast, path, fullpath) -> target where target := <compute-target> source
  111. 111. From Use to Declaration editor-resolve: (SimpleType(type), position, ast, path, fullpath) -> target where Entity(target,_) := <declaration-of> type editor-resolve: (ref@PageRef(x,e*), position, ast, path, fullpath) -> target where TemplateDef(_,target,_,_) := <declaration-of> ref
  112. 112. Experience
  113. 113. Experience Bootstrapped ★ SDF, Stratego, ATerm, PP ★ Editor service language Spoofax ★ WebDSL ★ Mobl: mobile web applications ★ Acoda: migration of data models ★ Aster: attribute grammars ★ PIL: platform independent language ★ Student languages SDF+Stratego ★ Java, AspectJ, XML, PHP, SQL, Jimple, Octave, Dot, ...
  114. 114. Future Work (in progress) Higher-level definition of scope & type system ★ cover scope systems of real languages Refactoring ★ layout preservation, generic refactoring strategies Systematic language design ★ how do we decide that a language design is good? Interaction design ★ higher-level specifications of interaction Combining textual and visual Textual IDE in the browser
  115. 115. - the end - Lennart C. L. Kats, Eelco Visser. The Spoofax Language Workbench. Rules for Declarative Specification of Languages and IDEs. OOPSLA 2010 http://spoofax.org http://webdsl.org http://mobl.org
  116. 116. Context-sensitive Transformation
  117. 117. define page blog(b : Blog) { Template Inlining header{output(b.name)} list{ for(p : Post in b.posts) { listitem{ outputPost(p) } } } } define outputPost(pst : Post) { navigate post(pst) { output(pst.name) } } define list() { <ul> elements </ul> } define listitem() { <li> elements </li> } define header() { <h1> elements </h1> } define page blog ( b : Blog ) { <h1> output(b.name) </h1> <ul> for ( p : Post in b.posts ) { <li> navigate post(p) { output(p.name) } </li> } </ul> }
  118. 118. Context-sensitive Transformation Local-to-local ★ replace term by another term Local-to-global ★ local term influence terms in other parts of model Global-to-local ★ global information influences transformation of local terms
  119. 119. define page blog(b : Blog) { header{output(b.name)} list{ for(p : Post in b.posts) { listitem{ outputPost(p) } } } Inlining is Global-to-Local } define outputPost(pst : Post) { navigate post(pst) { output(pst.name) } } define list() { <ul> elements </ul> } define listitem() { <li> elements </li> } define header() { <h1> elements </h1> } define page blog ( b : Blog ) { <h1> output(b.name) </h1> <ul> for ( p : Post in b.posts ) { <li> navigate post(p) { output(p.name) } </li> } </ul> }
  120. 120. Inlining as Rewrite Problem outputPost(p) -> navigate post(p) { output(p.name) } where define outputPost(pst : Post) { navigate post(pst) { output(pst.name) } } (this is not a valid Stratego rewrite rule)
  121. 121. Rewrite Rules are Context-free desugar : Property(x, t) -> Property(x, t, [])
  122. 122. Simplification: Parameterless Templates define page blog(b : Blog) { ... footer() } define footer() { navigate url(http://webdsl.org) { “WebDSL” } } define page blog ( b : Blog ) { ... container() { navigate url(http://webdsl.org) { “WebDSL” } } }
  123. 123. Rewrite in Context Inline : [def@TemplateDef(f,[],elem1*) | def1*] -> [def | def2*] where def2* := <alltd((Call(f) -> Call("container", elem1*)))> def1* local traversal bound in context container needed to replace single call with multiple elements
  124. 124. Intermezzo: Rewriting Strategies
  125. 125. transformation is partial* function from terms to terms * transformation may fail to apply to a term
  126. 126. Defining Transformations: Rules & Strategies Rewrite rules are basic transformations ★ transform term matching lhs to instantiation of rhs ★ evaluate condition (where clause) Strategies combine rules into complex transformations ★ select rules to apply ★ select algorithm to apply rules Strategy combinators ★ composition of custom transformations
  127. 127. Strategy Combinators id ★ Identity fail ★ failure s1 ; s2 ★ sequential composition s1 <+ s2 ★ choice
  128. 128. Variables {x, y : s} ★ term variable scope ?t ★ match term pattern t !t ★ build term pattern t t1 := t2 ★ match term t2 to term (pattern) t1 <s> t ★ apply strategy s to term t
  129. 129. Rewrite Rules l : t1 -> t2 where s ★ named, scoped rewrite rule ★ all variables in t1, t2, s are in scope of the rule (t1 -> t2 where s) ★ unscoped rewrite rule ★ variables are in scope of enclosing scope
  130. 130. Rewrite in Context Inline : [def@TemplateDef(f,[],elem1*) | def1*] -> [def | def2*] where def2* := <alltd((Call(f) -> Call("container", elem1*)))> def1* bound in context
  131. 131. Inline : [def@TemplateDef(f,[],elem*) | def1*] -> [def | def2*] where def2* := <alltd((Call(f) -> Call("container", elem*)))> def1* f elem* [ TemplateDef( "footer" , [] , [Navigate(PageRef("webdsl", []), [String("WebDSL")])] ) , TemplateDef( "blog" , [Param("b", SimpleType("Blog"))] , [Call("footer")] ) ] Call(f) def1*
  132. 132. Strategy Definitions f(x,y|a,b) = s ★ x, y are strategy parameters ★ a, b are term parameters ★ s uses all parameters f(x) = s f = s ★ term parameters are optional ★ all parameters are optional Examples ★ try(s) = s <+ id ★ repeat(s) = try(s; repeat(s))
  133. 133. Rules with Parameters Transform all elements of a list map(s) : [] -> [] map(s) : [x|xs] -> [<s>x | <map(s)> xs] Invert order of elements of a list inverse(|ys) : [] -> ys inverse(|ys) : [x|xs] -> <inverse(|[x|ys])> xs Pair elements of two lists zip(s) : ([],[]) -> [] zip(s) : ([x|xs],[y|ys]) -> [<s>(x,y) | <zip(s)>(xs,ys)]
  134. 134. Traversal Combinators all(s) ★ apply s to all direct sub-terms (children) one(s) ★ apply s to exactly one sub-term some(s) ★ apply s to at least one sub-term
  135. 135. Traversal Strategies topdown(s) = s; all(topdown(s)) ★ apply s to all sub-terms in top-down order bottomup(s) = all(bottomup(s)); s ★ apply s to all sub-terms in bottom-up order oncetd(s) = s <+ one(oncetd(s)) ★ apply s to one sub-term alltd(s) = s <+ all(alltd(s)) ★ apply s to frontier
  136. 136. Rewrite in Context: Local Traversal Inline : [def@TemplateDef(f,[],elem1*) | def1*] -> [def | def2*] where def2* := <alltd((Call(f) -> Call("container", elem1*)))> def1* local traversal
  137. 137. Dynamic Rewrite Rules
  138. 138. Rewrite in Context: Not Optimal Inline : [def@TemplateDef(f,[],elem1*) | def1*] -> [def | def2*] where def2* := <alltd((Call(f) -> Call("container", elem1*)))> def1* requires def before use local traversal for each declaration
  139. 139. Dynamic Rewrite Rules Inline : [def@TemplateDef(f,[],elem1*) | def1*] -> [def | def2*] where def2* := <alltd((Call(f) -> Call("container", elem1*)))> def1* declare-inline : TemplateDef(f,[],elem1*) -> TemplateDef(f,[],elem1*) where rules( InlineTemplate : Call(f) -> Call("container", elem1*) ) inline = alltd(declare-inline); topdown(try(InlineTemplate)) separate traversal from rule definition (binding closures)
  140. 140. Inlining as Rewrite Problem (Revisited) outputPost(p) -> navigate post(p) { output(p.name) } where define outputPost(pst : Post) { navigate post(pst) { output(pst.name) } } (informal) declare-inline : TemplateDef(f,[],elem1*) -> TemplateDef(f,[],elem1*) where rules( InlineTemplate : Call(f) -> Call("container", elem1*) ) (formal; but not yet complete)
  141. 141. define page blog(b : Blog) { Template Parameter header{output(b.name)} list{ for(p : Post in b.posts) { listitem{ outputPost(p) } } } } define outputPost(pst : Post) { navigate post(pst) { output(pst.name) } } define list() { <ul> elements </ul> } define listitem() { <li> elements </li> } define header() { <h1> elements </h1> } define page blog ( b : Blog ) { <h1> output(b.name) </h1> <ul> for ( p : Post in b.posts ) { <li> navigate post(p) { output(p.name) } </li> } </ul> }
  142. 142. Inlining Templates with Parameters declare-inline : def@TemplateDef(mod*,f,param*,elem1*) -> def where rules( InlineTemplate : Call(f, e*, []) -> Call("container", [], elem3*) where elem3* := <substitute> (param*, e*, elem1*) ) substitute : (param*, e*, elem1*) -> elem2* where {| Subst : <zip(bind-arg)> (param*, e*) ; elem2* := <alltd(Subst)> elem1* |} bind-arg : (Param(x, t), e) -> (Param(x, t), e) where rules( Subst : Var(x) -> e )
  143. 143. Element Parameters define list() { <ul> elements </ul> } declare-inline : def@TemplateDef(mod*,f,param*,elem1*) -> def where rules( InlineTemplate : Call(f, e*, elem2*) -> Call("container", [], elem3*) where {| Subst : rules( Subst : Elements() -> Call("container",[],elem2*) ) ; elem3* := <substitute> (param*, e*, elem1*) |} )
  144. 144. Removing Intermediate Structures rules // remove containers desugar-container : [Call("container",[], elem1*) | elem2*] -> [elem1*, elem2*] desugar : elem1* -> elem2* where elem2* := <at-suffix(desugar-container)> elem1* container needed to replace single call with multiple elements
  145. 145. Inlining Strategy module template-inlining imports libstratego-lib imports include/nwl imports desugar strategies inline-all = desugar-all; alltd(declare-inline); innermost(desugar <+ InlineTemplate) rules declare-inline : ...
  146. 146. module template-inlining imports libstratego-lib imports include/nwl imports desugar strategies inline-all = desugar-all; alltd(declare-inline); innermost(desugar <+ InlineTemplate) rules declare-inline : TemplateDef(mod*,f,param*,elem1*) -> TemplateDef(mod*,f,param*,elem1*) where rules( InlineTemplate : Call(f, e*, elem2*) -> Call("container", [], elem3*) Template Inlining where {| Subst : rules( Subst : Elements() -> Call("container", [], elem2*) ) ; elem3* := <substitute> (param*, e*, elem1*) Transformation ) |} substitute : (param*, e*, elem1*) -> elem2* where {| Subst : <zip(bind-arg)> (param*, e*) ; elem2* := <alltd(Subst)> elem1* |} bind-arg : (Param(x, t), e) -> (Param(x, t), e) where rules( Subst : Var(x) -> e ) rules // remove containers desugar-container : [Call("container",[], elem1*) | elem2*] -> [elem1*, elem2*] desugar : elem1* -> elem2* where elem2* := <at-suffix(desugar-container)> elem1*

×