This document discusses modularity and composition of language specifications. It describes how language extensions can be added to a "host" language in a modular way through tools like ableP. The key points are: 1) ableP allows users to choose extensions independently and composes them with the Promela host language to create a custom translator. 2) Developing extensible language frameworks requires addressing composable syntax and semantics through techniques like context-aware scanning and modular attribute grammar analyses. 3) Modular analyses like determinism and completeness checking ensure the composed language specification is well-defined and will result in a working translator. This allows extensions to be developed independently and composed automatically.

1. Modularity and Composition of Language
Specifications
Ted Kaminski, Lijesh Krishnan, Yogesh Mali, August
Schwerdfeger and Eric Van Wyk
University of Minnesota
September 17, 2012, Lund, Sweden
Thanks to the National Science Foundation (NSF), Funda¸˜o Luso-Americana
ca
(FLAD), McKnight Foundation, IBM, and the University of Minnsota for funding
portions of this work.
Page 1

2. Extensible Language Frameworks — ableP
add features to a “host” language — Promela
new language constructs - their syntax and semantics
select (altitude: 1000 .. 10000);
select (altitude: 1000 .. 10000 step 100);
select (altQuality: High, Med, Low);
DTSpin constructs: timer t; t = 1; expire(t);
new semantics of existing constructs
semantic analysis, translations to new target languages, ...
type checking
advanced ETCH-style type inference and checking
Page 2

3. Various means for extending Promela
select (v: 1 .. 10) added in SPIN version 6.
DTSPIN features
as CPP macros — lightweight
or modifying the SPIN implementation — heavyweight
ETCH, enhanced type checking
built their own scanner and parser using SableCC
ableP — middleweight approach
Page 3

4. An example
An altitude switch model that uses
enhanced select statements
DTSPIN-like constructs
tabular Boolean expressions (` la RSML and SCR)
a
An instance of ableP parses and analyzes the model, then
generates its translation to pure Promela.
% java -jar ableP.aviation.jar AltSwitch.xpml
% spin -a AltSwitch.pml
Page 4

5. Our approach:
Users choose (independently developed) extensions.
Tools compose the extensions and Promela host language.
Distinguish
1. extension user
has no knowledge of language design or implementations
2. extension developer
must know about language design and implementation
1. Tools and formalisms support automatic composition.
2. Modular analyses ensure the composition results in a working
translator.
Value easy composition over expressivity, accept some
restrictions
on syntax
new constructs are translated to “pure” Promela
ableP “instances” are smart pre-processors
Page 5

6. Extending ableP with independently developed extensions
Extension user directs underlying tools to
compose chosen extensions and the host language
to create a custom translator for the extended language
Silver grammar modules define sets of specifications
composition is set union, order does not matter
Consider the Silver specification for this composition.
Page 6

7. Developing language extensions
Two primary challenges:
1. composable syntax — enables building a parser
context-aware scanning [GPCE’07]
modular determinism analysis [PLDI’09]
Copper
2. composable semantics — analysis and translations
attribute grammars with forwarding, collections and
higher-order attributes
set union of specification components
sets of productions, non-terminals, attributes
sets of attribute defining equations, on a production
sets of equations contributing values to a single attribute
modular well-definedness analysis [SLE’12a]
monolithic termination analysis [SLE’12b]
Silver
Page 7

8. Context aware scanning
Scanner recognizes only tokens valid for current “context”
keeps embedded sub-languages, in a sense, separate
Consider:
chan in, out;
for i in a { a[i] = i*i ; }
Two terminal symbols that match “in”.
terminal IN ’in’ ;
terminal ID /[a-zA-Z ][a-zA-Z 0-9]*/
submits to {promela kwd };
terminal FOR ’for’ lexer classes {promela kwd };
Page 8

9. Allows parsing of embedded C code
c_decl {
typedef struct Coord {
int x, y; } Coord; }
c_state "Coord pt" "Global" /* goes in state vector */
int z = 3; /* standard global decl */
active proctype example()
{ c_code { now.pt.x = now.pt.y = 0; };
do :: c_expr { now.pt.x == now.pt.y }
-> c_code { now.pt.y++; }
:: else -> break
od;
c_code { printf("values %d: %d, %d,%dn",
Pexample->_pid, now.z, now.pt.x, now.pt.y);
};
}
Page 9

10. Semantics for host language assignment constructs
grammar edu:umn:cs:melt:ableP:host:core:abstractsyntax;
abstract production defaultAssign
s::Stmt ::= lhs::Expr rhs::Expr
{ s.pp = lhs.pp ++ " = " ++ rhs.pp ++ " ;n" ;
lhs.env = s.env; rhs.env = s.env;
s.defs = emptyDefs();
s.errors := lhs.errors ++ rhs.errors ;
}
Adding extension constructs involves writing similar productions.
Page 10

11. Adding ETCH-like semantic analysis.
grammar edu:umn:cs:melt:ableP:extensions:typeChecking ;
synthesized attribute typerep::TypeRep
occurs on Expr, Decls ;
aspect production varRef
e::Expr ::= id::ID
{ e.typerep = ... retrieve from declaration
found in e.env ... ; }
aspect production defaultAssign
s::Stmt ::= lhs::Expr rhs::Expr
{ s.errors <- if isCompatible(lhs.typerep, rhs.typerep)
then [ ]
else [ mkError ("Incompatible types ...") ];
}
Page 11

12. Extensibility: safe composability
New attributes
New productions
Host
Ext 2
Ext 1 Problem
independent
extensions
Page 12

13. Extensibility: safe composability
Page 13

14. Extensions get undefined semantics from host translation.
grammar edu:umn:cs:melt:ableP:extensions:enhancedSelect ;
abstract production selectFrom
s::Stmt ::= sl::’select’ v::Expr es::Exprs
{
s.pp = "select ( " ++ v.pp ++ ":" ++ es.pp ++ " ); n" ;
s.errors := v.errors ++ es.errors ++
if ... check that all expressions in ’es’ have
same type as ’v’ ...
then [ mkError ("Error: select statement " ++
"requires same type ... ") ]
else [ ] ;
forwards to ifStmt( mkOptions (v, es) ) ;
}
Page 14

15. Modular analysis
Ensuring that the composition will be successful.
Page 15

16. Context free grammars
1 2 i
GH ∪ GE ∪ GE ∪ ... ∪ GE
∪ of sets of nonterminals, terminals, productions
Composition of all is an context free grammar.
Is it non-ambiguous, useful for deterministic (LR) parsing?
1
conflictFree(GH ∪ GE ) holds
2
conflictFree(GH ∪ GE ) holds
i
conflictFree(GH ∪ GE ) holds
1 2 i
conflictFree(GH ∪ GE ∪ GE ∪ ... ∪ GE ) may not hold
Page 16

17. Attribute grammars
1 2 i
AGH ∪ AGE ∪ AGE ∪ ... ∪ AGE
∪ of sets of attributes, attribute equations, occurs-on
declarations
Composition of all is an attribute grammar.
Completeness: ∀ production, ∀ attribute, ∃ an equation
1
complete(AGH ∪ AGE ) holds
2
complete(AGH ∪ AGE ) holds
i
complete(AGH ∪ AGE ) holds
1 2 i
complete(AGH ∪ AGE ∪ AGE ∪ ... ∪ AGE ) may not hold
similarly for non-circularity of the AG
Page 17

18. Detecting problems, ensuring composition
When can some analysis of the language specification be applied?
When ...
1. the host language is developed ?
2. a language extensions is developed ?
3. when the host and extensions are composed ?
4. when the resulting language tools are run ?
Page 18

19. Libraries, and modular type checking
Libraries “just work”
Type checking is done by the library writer, modularly.
Language extensions should be like libraries, composition of
“verified” extensions should “just work.”
Page 19

20. Modular determinism analysis for grammars, 2009
1 2 i
GH ∪ GE ∪ GE ∪ ... ∪ GE
1 1
isComposable(GH , GE ) ∧ conflictFree(GH ∪ GE ) holds
2 2
isComposable(GH , GE ) ∧ conflictFree(GH ∪ GE ) holds
i i
isComposable(GH , GE ) ∧ conflictFree(GH ∪ GE ) holds
1 2
these imply conflictFree(GH ∪ GE ∪ GE ∪ ...) holds
i
( ∀i ∈ [1, n].isComposable(GH , GE ) ∧
i
conflictFree(GH ∪ {GE )} )
1 n
=⇒ conflictFree(GH ∪ {GE , . . . , GE })
Some restrictions to extension introduced syntax apply, of
course.
Page 20

21. Modular completeness analysis for attribute grammars
1 2 i
AGH ∪ AGE ∪ AGE ∪ ... ∪ AGE
1
modComplete(AGH ∪ AGE ) holds
2
modComplete(AGH ∪ AGE ) holds
i
modComplete(AGH ∪ AGE ) holds
1 2
these imply complete(AGH ∪ AGE ∪ AGE ∪ ...) holds
(∀i ∈ [1, n].modComplete(AG H , AG iE ))
1 n
=⇒ complete(AG H ∪ {AGE , ..., AGE }).
similarly for non-circularity of the AG
Again, some restrictions on extensions.
Page 21

22. Modular completeness analysis
The analysis modComplete is defined as follows:
modComplete(AG H , AG E )
noOrphanOccursOn(AG H , AG E ) ∧
noOrphanAttrEqs(AG H , AG E ) ∧
noOrphanProds(AG H , AG E ) ∧
synComplete(AG H , AG E ) ∧
modularFlowTypes(flowTypes(AG H ),
flowTypes(AG H ∪ AG E )) ∧
inhComplete(AG , AG E , flowTypes(AG H ∪ AG E ))
H
Some “structural” requirements, some flow-type requirements
Silver’s module system prevents duplicate nonterminal,
attribute, production declarations.
Page 22

23. No orphan occurs on declarations
“an AG declares a occurs on nt only if it declares or exports a
or nt”
noOrphanOccursOn(AG H , AG E )
holds if and only if each occurs-on declaration
“attribute a occurs on nt” in AG H ∪ AG E
is exported by the grammar declaring a or the grammar
declaring nt.
Page 23

24. No orphan attribute equations
“an AG provides an equation n.a=e on p with l.h.s. nonterminal
nt only if it declares/exports the production p or the occurs-on
declaration a occurs on nt.”
noOrphanAttrEqs(AG H , AG E )
holds if and only if each equation n.a = e in a production p is
exported by the grammar declaring the (non-aspect)
production p or the grammar declaring the occurs-on
declaration “attribute a occurs on nt” (where n has type
nt.)
Page 24

25. No orphan production declarations
“Productions in extensions that build host language nonterminals
must forward.”
noOrphanProds(AG H , AG E )
holds if and only if for each production declaration p in
AG H ∪ AG E with left hand side nonterminal nt, the
production p is either exported by the grammar declaring nt,
or p forwards.
Page 25

26. Completeness of synthesized equations
“A production forwards or has equations for all its attributes, these
may be on aspect productions.”
synComplete(AG H , AG E )
holds if and only if for each occurs-on declaration
attribute a occurs on nt, and for each non-forwarding
production p that constructs nt, there exists a rule defining
the synthesized equation p : x.a, where x is the left hand side
of the production.
Page 26

27. Flow Types
A flow-type captures how information flow between attributes.
For a non-terminal, it maps synthesized attributes to the
inherited attributes on which it depends.
ftnt :: As → 2AI
e.g. ftExpr (type) = {env }
Page 27

28. Modularity of flow types
“Host language attributes on host language nonterminals do not
depend on extension-declared inherited attributes.”
modularFlowTypes(flowTypes(AG H ),
flowTypes(AG H ∪ AG E ))
holds if and only if for each ft H ∈ flowTypes(AG H ) and
nt
ft H∪E ∈ flowTypes(AG H ∪ AG E ),
nt
for all synthesized attributes s and for all nonterminals nt such
that attribute s occurs on nt is declared in AG H ,
ft H∪E (s) ⊆ ft H (s).
nt nt
Page 28

29. Effective completeness of inherited equations
“All inherited attributes required to compute a synthesized
attribute on a node have defining equations.”
inhComplete(AG H , AG E , flowTypes(AG H ∪ AG E ))
holds if and only if for every production p in AG H ∪ AG E and
for every access to a synthesized attribute n.s in an expression
within p (where n has type nt,) and for each inherited
attribute i ∈ ftnt (s), there exists an equation n.i = e for p.
Page 29

30. A generalization
Silver does not identify “host” and “extensions.”
The analysis works on import grammar relationships.
A grammar A that imports B is seen as an extension to B.
So, Silver libraries are hosts, in essence.
Page 30

31. Evaluation
Q: Are these restrictions too overbearing?
A1: No, they are turned on by default.
A2: No, except for reference attributes.
Applied to Silver compiler’s Silver specs
1. We found a few bugs.
2. We moved some declarations to new grammars.
These were bad design “smells.”
3. We extended Silver
Annotations allowing host language to be modularized without
respect to the rules.
These treat the “whole” host language, spread across many
modules, as one for the analysis.
Default attributes.
Reference attributes - rather severe restrictions.
All inherited attributes must be provided and no more can be
added.
Page 31

32. Modular circularity analysis
Analysis extends to check for no cycles, modularly.
Instead of a flow type for each non-terminal, there is a set of
flow-graphs for each.
Analysis ensures no new patterns of information flow are
added by an extension.
Page 32

33. Lessons learned ?
For extensible language frameworks
Analysis - modular or don’t bother?
But my undergrads like static completeness detection.
Page 33

34. Termination analysis for Higher Order AGs
detects potential creation of an infinite number of trees.
also a restricted modular version.
This, with modular well-definedness analysis, ensures that the
composed AG will terminate normally.
Page 34

35. So ... for ableP
ableP supports the simple composition of language extensions
This creates translators and analyzers for customized
Promela-based languages.
extensions can be verified to compose, with other verified
extensions — done by extension developers
adding (independently developed) extensions that add new
features and new analysis on host features is supported
Page 35

36. Some questions ... what to call these?
We’ve called this a “modular” analysis.
Maybe “static composition analysis”
analysis that happens before composition
As opposed to “dynamic composition analysis”
analysis that happens during composition
But “static” and “dynamic” suggest 2 points in time
before and during run-time
We have more than 2 interesting points in time.
1. host development, 2. extension development,
3. composition, 4. translation, 5. execution
Page 36

37. Future work
These modular analyses, for syntax and static semantics,
ensure the generated translator terminates normally.
They don’t ensure that the translator is correct.
Host language specific invariants are not enforced.
How can we do that?
Page 37

38. Thanks for your attention.
Questions?
http://melt.cs.umn.edu/
evw@cs.umn.edu
Page 38

39. [GPCE’07] Eric Van Wyk and August Schwerdfeger.
Context-aware scanning for parsing extensible
languages.
In Intl. Conf. on Generative Programming and
Component Engineering, (GPCE). ACM Press,
October 2007.
[PLDI’09] August Schwerdfeger and Eric Van Wyk.
Verifiable composition of deterministic grammars.
In Proc. of ACM SIGPLAN Conference on
Programming Language Design and Implementation
(PLDI). ACM Press, June 2009.
Page 39

40. [SLE’12a] Ted Kaminski and Eric Van Wyk.
Modular well-definedness analysis for attribute
grammars.
In Proceedings of 5th the International Conference
on Software Language Engineering (SLE 2012),
LNCS. Springer-Verlag, September 2012.
To appear.
[SLE’12b] Lijesh Krishnan and Eric Van Wyk.
Termination analysis for higher-order attribute
grammars.
In Proceedings of 5th the International Conference
on Software Language Engineering (SLE 2012),
LNCS. Springer-Verlag, September 2012.
To appear.
Page 40