Talk Lund University CS Department

Uploaded on

Slides from my talk on extensible languages and modularity and composition in language specifications. I gave this talk on September 17 at the Computer Science Department at Lund University, Lund, …

Slides from my talk on extensible languages and modularity and composition in language specifications. I gave this talk on September 17 at the Computer Science Department at Lund University, Lund, Sweden.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Modularity and Composition of Language Specifications Ted Kaminski, Lijesh Krishnan, Yogesh Mali, August Schwerdfeger and Eric Van Wyk University of Minnesota September 17, 2012, Lund, SwedenThanks to the National Science Foundation (NSF), Funda¸˜o Luso-Americana ca(FLAD), McKnight Foundation, IBM, and the University of Minnsota for fundingportions of this work. Page 1
  • 2. Extensible Language Frameworks — ableP add features to a “host” language — Promela new language constructs - their syntax and semantics select (altitude: 1000 .. 10000); select (altitude: 1000 .. 10000 step 100); select (altQuality: High, Med, Low); DTSpin constructs: timer t; t = 1; expire(t); new semantics of existing constructs semantic analysis, translations to new target languages, ... type checking advanced ETCH-style type inference and checking Page 2
  • 3. Various means for extending Promela select (v: 1 .. 10) added in SPIN version 6. DTSPIN features as CPP macros — lightweight or modifying the SPIN implementation — heavyweight ETCH, enhanced type checking built their own scanner and parser using SableCC ableP — middleweight approach Page 3
  • 4. An example An altitude switch model that uses enhanced select statements DTSPIN-like constructs tabular Boolean expressions (` la RSML and SCR) a An instance of ableP parses and analyzes the model, then generates its translation to pure Promela. % java -jar ableP.aviation.jar AltSwitch.xpml % spin -a AltSwitch.pml Page 4
  • 5. Our approach: Users choose (independently developed) extensions. Tools compose the extensions and Promela host language. Distinguish 1. extension user has no knowledge of language design or implementations 2. extension developer must know about language design and implementation 1. Tools and formalisms support automatic composition. 2. Modular analyses ensure the composition results in a working translator. Value easy composition over expressivity, accept some restrictions on syntax new constructs are translated to “pure” Promela ableP “instances” are smart pre-processors Page 5
  • 6. Extending ableP with independently developed extensions Extension user directs underlying tools to compose chosen extensions and the host language to create a custom translator for the extended language Silver grammar modules define sets of specifications composition is set union, order does not matter Consider the Silver specification for this composition. Page 6
  • 7. Developing language extensions Two primary challenges: 1. composable syntax — enables building a parser context-aware scanning [GPCE’07] modular determinism analysis [PLDI’09] Copper 2. composable semantics — analysis and translations attribute grammars with forwarding, collections and higher-order attributes set union of specification components sets of productions, non-terminals, attributes sets of attribute defining equations, on a production sets of equations contributing values to a single attribute modular well-definedness analysis [SLE’12a] monolithic termination analysis [SLE’12b] Silver Page 7
  • 8. Context aware scanning Scanner recognizes only tokens valid for current “context” keeps embedded sub-languages, in a sense, separate Consider: chan in, out; for i in a { a[i] = i*i ; } Two terminal symbols that match “in”. terminal IN ’in’ ; terminal ID /[a-zA-Z ][a-zA-Z 0-9]*/ submits to {promela kwd }; terminal FOR ’for’ lexer classes {promela kwd }; Page 8
  • 9. Allows parsing of embedded C code c_decl { typedef struct Coord { int x, y; } Coord; } c_state "Coord pt" "Global" /* goes in state vector */ int z = 3; /* standard global decl */ active proctype example() { c_code { = = 0; }; do :: c_expr { == } -> c_code {; } :: else -> break od; c_code { printf("values %d: %d, %d,%dn", Pexample->_pid, now.z,,; }; } Page 9
  • 10. Semantics for host language assignment constructs grammar edu:umn:cs:melt:ableP:host:core:abstractsyntax; abstract production defaultAssign s::Stmt ::= lhs::Expr rhs::Expr { s.pp = lhs.pp ++ " = " ++ rhs.pp ++ " ;n" ; lhs.env = s.env; rhs.env = s.env; s.defs = emptyDefs(); s.errors := lhs.errors ++ rhs.errors ; } Adding extension constructs involves writing similar productions. Page 10
  • 11. Adding ETCH-like semantic analysis. grammar edu:umn:cs:melt:ableP:extensions:typeChecking ; synthesized attribute typerep::TypeRep occurs on Expr, Decls ; aspect production varRef e::Expr ::= id::ID { e.typerep = ... retrieve from declaration found in e.env ... ; } aspect production defaultAssign s::Stmt ::= lhs::Expr rhs::Expr { s.errors <- if isCompatible(lhs.typerep, rhs.typerep) then [ ] else [ mkError ("Incompatible types ...") ]; } Page 11
  • 12. Extensibility: safe composability New attributes New productions Host Ext 2 Ext 1 Problem independent extensions Page 12
  • 13. Extensibility: safe composability Page 13
  • 14. Extensions get undefined semantics from host translation. grammar edu:umn:cs:melt:ableP:extensions:enhancedSelect ; abstract production selectFrom s::Stmt ::= sl::’select’ v::Expr es::Exprs { s.pp = "select ( " ++ v.pp ++ ":" ++ es.pp ++ " ); n" ; s.errors := v.errors ++ es.errors ++ if ... check that all expressions in ’es’ have same type as ’v’ ... then [ mkError ("Error: select statement " ++ "requires same type ... ") ] else [ ] ; forwards to ifStmt( mkOptions (v, es) ) ; } Page 14
  • 15. Modular analysis Ensuring that the composition will be successful. Page 15
  • 16. Context free grammars 1 2 i GH ∪ GE ∪ GE ∪ ... ∪ GE ∪ of sets of nonterminals, terminals, productions Composition of all is an context free grammar. Is it non-ambiguous, useful for deterministic (LR) parsing? 1 conflictFree(GH ∪ GE ) holds 2 conflictFree(GH ∪ GE ) holds i conflictFree(GH ∪ GE ) holds 1 2 i conflictFree(GH ∪ GE ∪ GE ∪ ... ∪ GE ) may not hold Page 16
  • 17. Attribute grammars 1 2 i AGH ∪ AGE ∪ AGE ∪ ... ∪ AGE ∪ of sets of attributes, attribute equations, occurs-on declarations Composition of all is an attribute grammar. Completeness: ∀ production, ∀ attribute, ∃ an equation 1 complete(AGH ∪ AGE ) holds 2 complete(AGH ∪ AGE ) holds i complete(AGH ∪ AGE ) holds 1 2 i complete(AGH ∪ AGE ∪ AGE ∪ ... ∪ AGE ) may not hold similarly for non-circularity of the AG Page 17
  • 18. Detecting problems, ensuring composition When can some analysis of the language specification be applied? When ... 1. the host language is developed ? 2. a language extensions is developed ? 3. when the host and extensions are composed ? 4. when the resulting language tools are run ? Page 18
  • 19. Libraries, and modular type checking Libraries “just work” Type checking is done by the library writer, modularly. Language extensions should be like libraries, composition of “verified” extensions should “just work.” Page 19
  • 20. Modular determinism analysis for grammars, 2009 1 2 i GH ∪ GE ∪ GE ∪ ... ∪ GE 1 1 isComposable(GH , GE ) ∧ conflictFree(GH ∪ GE ) holds 2 2 isComposable(GH , GE ) ∧ conflictFree(GH ∪ GE ) holds i i isComposable(GH , GE ) ∧ conflictFree(GH ∪ GE ) holds 1 2 these imply conflictFree(GH ∪ GE ∪ GE ∪ ...) holds i ( ∀i ∈ [1, n].isComposable(GH , GE ) ∧ i conflictFree(GH ∪ {GE )} ) 1 n =⇒ conflictFree(GH ∪ {GE , . . . , GE }) Some restrictions to extension introduced syntax apply, of course. Page 20
  • 21. Modular completeness analysis for attribute grammars 1 2 i AGH ∪ AGE ∪ AGE ∪ ... ∪ AGE 1 modComplete(AGH ∪ AGE ) holds 2 modComplete(AGH ∪ AGE ) holds i modComplete(AGH ∪ AGE ) holds 1 2 these imply complete(AGH ∪ AGE ∪ AGE ∪ ...) holds (∀i ∈ [1, n].modComplete(AG H , AG iE )) 1 n =⇒ complete(AG H ∪ {AGE , ..., AGE }). similarly for non-circularity of the AG Again, some restrictions on extensions. Page 21
  • 22. Modular completeness analysis The analysis modComplete is defined as follows: modComplete(AG H , AG E ) noOrphanOccursOn(AG H , AG E ) ∧ noOrphanAttrEqs(AG H , AG E ) ∧ noOrphanProds(AG H , AG E ) ∧ synComplete(AG H , AG E ) ∧ modularFlowTypes(flowTypes(AG H ), flowTypes(AG H ∪ AG E )) ∧ inhComplete(AG , AG E , flowTypes(AG H ∪ AG E )) H Some “structural” requirements, some flow-type requirements Silver’s module system prevents duplicate nonterminal, attribute, production declarations. Page 22
  • 23. No orphan occurs on declarations “an AG declares a occurs on nt only if it declares or exports a or nt” noOrphanOccursOn(AG H , AG E ) holds if and only if each occurs-on declaration “attribute a occurs on nt” in AG H ∪ AG E is exported by the grammar declaring a or the grammar declaring nt. Page 23
  • 24. No orphan attribute equations “an AG provides an equation n.a=e on p with l.h.s. nonterminal nt only if it declares/exports the production p or the occurs-on declaration a occurs on nt.” noOrphanAttrEqs(AG H , AG E ) holds if and only if each equation n.a = e in a production p is exported by the grammar declaring the (non-aspect) production p or the grammar declaring the occurs-on declaration “attribute a occurs on nt” (where n has type nt.) Page 24
  • 25. No orphan production declarations “Productions in extensions that build host language nonterminals must forward.” noOrphanProds(AG H , AG E ) holds if and only if for each production declaration p in AG H ∪ AG E with left hand side nonterminal nt, the production p is either exported by the grammar declaring nt, or p forwards. Page 25
  • 26. Completeness of synthesized equations “A production forwards or has equations for all its attributes, these may be on aspect productions.” synComplete(AG H , AG E ) holds if and only if for each occurs-on declaration attribute a occurs on nt, and for each non-forwarding production p that constructs nt, there exists a rule defining the synthesized equation p : x.a, where x is the left hand side of the production. Page 26
  • 27. Flow Types A flow-type captures how information flow between attributes. For a non-terminal, it maps synthesized attributes to the inherited attributes on which it depends. ftnt :: As → 2AI e.g. ftExpr (type) = {env } Page 27
  • 28. Modularity of flow types “Host language attributes on host language nonterminals do not depend on extension-declared inherited attributes.” modularFlowTypes(flowTypes(AG H ), flowTypes(AG H ∪ AG E )) holds if and only if for each ft H ∈ flowTypes(AG H ) and nt ft H∪E ∈ flowTypes(AG H ∪ AG E ), nt for all synthesized attributes s and for all nonterminals nt such that attribute s occurs on nt is declared in AG H , ft H∪E (s) ⊆ ft H (s). nt nt Page 28
  • 29. Effective completeness of inherited equations “All inherited attributes required to compute a synthesized attribute on a node have defining equations.” inhComplete(AG H , AG E , flowTypes(AG H ∪ AG E )) holds if and only if for every production p in AG H ∪ AG E and for every access to a synthesized attribute n.s in an expression within p (where n has type nt,) and for each inherited attribute i ∈ ftnt (s), there exists an equation n.i = e for p. Page 29
  • 30. A generalization Silver does not identify “host” and “extensions.” The analysis works on import grammar relationships. A grammar A that imports B is seen as an extension to B. So, Silver libraries are hosts, in essence. Page 30
  • 31. Evaluation Q: Are these restrictions too overbearing? A1: No, they are turned on by default. A2: No, except for reference attributes. Applied to Silver compiler’s Silver specs 1. We found a few bugs. 2. We moved some declarations to new grammars. These were bad design “smells.” 3. We extended Silver Annotations allowing host language to be modularized without respect to the rules. These treat the “whole” host language, spread across many modules, as one for the analysis. Default attributes. Reference attributes - rather severe restrictions. All inherited attributes must be provided and no more can be added. Page 31
  • 32. Modular circularity analysis Analysis extends to check for no cycles, modularly. Instead of a flow type for each non-terminal, there is a set of flow-graphs for each. Analysis ensures no new patterns of information flow are added by an extension. Page 32
  • 33. Lessons learned ? For extensible language frameworks Analysis - modular or don’t bother? But my undergrads like static completeness detection. Page 33
  • 34. Termination analysis for Higher Order AGs detects potential creation of an infinite number of trees. also a restricted modular version. This, with modular well-definedness analysis, ensures that the composed AG will terminate normally. Page 34
  • 35. So ... for ableP ableP supports the simple composition of language extensions This creates translators and analyzers for customized Promela-based languages. extensions can be verified to compose, with other verified extensions — done by extension developers adding (independently developed) extensions that add new features and new analysis on host features is supported Page 35
  • 36. Some questions ... what to call these? We’ve called this a “modular” analysis. Maybe “static composition analysis” analysis that happens before composition As opposed to “dynamic composition analysis” analysis that happens during composition But “static” and “dynamic” suggest 2 points in time before and during run-time We have more than 2 interesting points in time. 1. host development, 2. extension development, 3. composition, 4. translation, 5. execution Page 36
  • 37. Future work These modular analyses, for syntax and static semantics, ensure the generated translator terminates normally. They don’t ensure that the translator is correct. Host language specific invariants are not enforced. How can we do that? Page 37
  • 38. Thanks for your attention. Questions? Page 38
  • 39. [GPCE’07] Eric Van Wyk and August Schwerdfeger. Context-aware scanning for parsing extensible languages. In Intl. Conf. on Generative Programming and Component Engineering, (GPCE). ACM Press, October 2007.[PLDI’09] August Schwerdfeger and Eric Van Wyk. Verifiable composition of deterministic grammars. In Proc. of ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM Press, June 2009. Page 39
  • 40. [SLE’12a] Ted Kaminski and Eric Van Wyk. Modular well-definedness analysis for attribute grammars. In Proceedings of 5th the International Conference on Software Language Engineering (SLE 2012), LNCS. Springer-Verlag, September 2012. To appear.[SLE’12b] Lijesh Krishnan and Eric Van Wyk. Termination analysis for higher-order attribute grammars. In Proceedings of 5th the International Conference on Software Language Engineering (SLE 2012), LNCS. Springer-Verlag, September 2012. To appear. Page 40