ITU - MDD - XText

ITU - MDD – XText This presentation describes the use of XText. This presentation assumes a good knowledge of Data Modeling and Grammars as previously presented. This presentation is developed for MDD 2010 course at ITU, Denmark.

Parse trees are very detailed:
every step in a derivation is a node
After the parsing phase is done, the details of derivation are not needed for later phases
Semantic Analyzer removes intermediate productions to create an (abstract) syntax tree – known as an AST
Abstract Syntax Trees expr term factor ID: x expr ID: x Parse Tree: Abstract Syntax Tree:

Parsing an Expression assignment => ID “=“ expression; expression => expression “+” term | expression “-” term | term term => term “*” factor | term “/” factor | factor factor => “(“ expression “)” | ID | NUMBER assignment ~ y = (2*x + 5)*x - 7 factor ~ 7 term ~ (2*x + 5)*x NUMBER ~ 7 term ~ (2*x + 5) factor ~ x expression ~ 2*x + 5 ID ~ x factor ~ 2 NUMBER ~ 2 ID ~ y expresssion ~ (2*x + 5)*x - 7 expression ~ (2*x + 5)*x term ~ 7 ID ~ x factor ~ (2*x + 5) expression ~ 2*x term ~ 5 factor ~ 5 term ~ 2*x NUMBER ~ 5 factor ~ x term ~ 2

Building a AST assignment => ID “=“ expression; expression => expression “+” term | expression “-” term | term term => term “*” factor | term “/” factor | factor factor => “(“ expression “)” | ID | NUMBER assignment ~ y = (2*x + 5)*x - 7 factor ~ 7 term ~ (2*x + 5)*x NUMBER ~ 7 term ~ (2*x + 5) factor ~ x expression ~ 2*x + 5 ID ~ x factor ~ 2 NUMBER ~ 2 Assignment(y) Expression(-) Expression(*) NUMBER(7) Expression(+) Expression(*) NUMBER(2) NUMBER ID(x) ID(x) ID ~ y expresssion ~ (2*x + 5)*x - 7 expression ~ (2*x + 5)*x term ~ 7 ID ~ x factor ~ (2*x + 5) expression ~ 2*x term ~ 5 factor ~ 5 term ~ 2*x NUMBER ~ 5 factor ~ x term ~ 2

Level of Details
How detailed should a grammar be?
Should the grammar be rich – i.e. contain all details?
Or should the grammar be as thin as possible?
In general use a thin grammar to avoid too many keywords
For XText, use a rich grammar as this is used to provide context assist
Other similar questions:
Should you model dates?
How about ranges?
Flight : 'flight' ID '{' ( ('from'|'to') '=' STRING ';' )* '}' Flight : 'flight' ID '{' ( ID '=' STRING ';' )* '}'

Line feed or Explicit Terminators
Should you have an explicit “statement” terminator or use line feeds?
Statement Terminator such as “;” or “.”
Pro: much easier to detect errors in the input
Con: normally not natural for the user
Statement Terminator such as line feed
Pro: much easier to detect errors in the input
Con: cannot divide a logical line over multiple physical lines
No Statement Terminators
Pro: very natural for many users
Con: difficult to detect errors in the input
In general use a statement terminator – not a statement separator!
In our scenario, it properly makes good sense to use line feeds
And indentions for scoping

What is XText exactly
Xtext is a complete environment for development of textual programming languages and domain-specific languages .
It is implemented in Java and is based on Eclipse , EMF , and Antlr .
The Basic Idea?
Augment an EBNF grammar
Roll it through XText
Have an Eclipse Editor
The XText tool bench
Grammar is defined in an EBNF-like format in the Xtext editor.
The editor provides code completion and constraint checking for the grammars themselves
Grammar is a collection of Rules. Rules start with their name followed by “:” and ending with “;”

Getting Started with XText
Create a new XText project with the New… wizard

The XText File Structure
Three projects!
You can make changes to the files in the src files, but not the files in the src-gen folders
Your grammar rules Workflow description used to create editor Generated ECore model Generated ECore classes

An XText Model File ( .xtext )
Identify of model
Include of base declarations and terminals
Directive to create model
NS URI of model
It is also possible to import an existing model
grammar org.xtext.example.MyDsl with org.eclipse.xtext.common.Terminals generate myDsl "http://www.xtext.org/example/MyDsl" Model : Type*; …

Generating Files from a Model
To generate an Eclipse editor from a a model ( .xtext )
Select the workflow file ( .mwe 2 )
Use “Run As…”  “MWE2 WorkFlow”

The Basic XText Concepts
The AST is a ECore model
The rules of the input grammar are used to create the AST (more or less automatically) or interface to an existing imported model (=AST)
So we must identify
The entities of the model
The attributes of the model
The relations – containment and references – of the model
The input language of XText is a augmented EBNF (almost  )

Example
Grammar for a (very) simple type system
Model : Type*; Type: SimpleType | Entity; SimpleType: ‘ data type' ID; Entity : 'entity' ID ('extends' ID)? '{' Property* '}'; Property: 'property' ID ':' ID ('[]')?; datatype A datatype B entity E { property a : A property b : B[] } entity F extends E { property c : A } Model Type SimpleType Entity Property * * Super-type

XText rules are EBNF plus type information
Type Rules
For each rule XText creates an entity in the logical model
Each rule property results in a property in the logical model
String Rules
String rules are parsed to be a string
These are in effect custom lexer rules as they recognize string patterns
Enum Rules
Limited set of alternatives
Mapped to an enumeration in the logical model
Native Rules
A lexer rule that is mapped directly to an ANTLR rule
Mapped to a String
Different Kinds of XText Rules

Class of type rule defaults to name of rule 
Can be overruled via “{ClassName}” construct
A type rule can be abstract
An abstract type rule is basically a collection of OR-ed alternatives: R1 | R2 | R3
Mapped to an abstract metaclass
The or-ed alternatives become concrete subclasses
Common properties of the alternatives are lifted into the abstract superclass
Abstract Type Rules Model : Type*; Type: SimpleType | Entity; SimpleType: ‘ data type' ID; Entity : 'entity' ID ('extends' ID)? '{' Property* '}'; Property: 'property' ID ':' ID ('[]')?;

Example with Type Rules
Every rule corresponds to en entity
“ Or” rules becomes abstract classes with the entities of the sub-rules as child classes
Above we have Type as an abstract super class for SimpleType and Entity
Model : Type*; Type: SimpleType | Entity; SimpleType: 'type' ID; Entity : 'entity' ID ('extends' ID)? '{' Property* '}'; Property: 'property' ID ':' ID ('[]')?;

Some built-in String Rules (Terminals)
The definition of the central terminals
Really part of lexical analysis!
Specific terminals can be “hidden” in the grammar or for specific rules, meaning they are recognized but ignored during parsing…
Very useful if you want a line-oriented grammar
terminal ID : '^'?('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*; terminal INT returns ecore::EInt: ('0'..'9')+; terminal STRING : '"' ( 'apos; ('b'|'t'|'n'|'f'|'r'|'"'|"'"|'apos;) | !('apos;|'"') )* '"' | "'" ( 'apos; ('b'|'t'|'n'|'f'|'r'|'"'|"'"|'apos;) | !('apos;|"'") )* "'" ; terminal ML_COMMENT : '/*' -> '*/'; terminal SL_COMMENT : '//' !(' '|' ')* (' '? ' ')?; terminal WS : (' '|' '|' '|' ')+;

A type rule has a rule composition – made of a number of elements
It may contain keywords (using string literal syntax)
It also contains properties which will result in properties of the type class
The property type is derived from the called rule
There are different kinds of properties
= (single assign)
+= (multiple assign/add)
?= (boolean assign)
Properties (Attributes and Relations) in Type Rules
There are different property cardinalities
? (0..1)
* (0..n)
+ (1..n)
nothing (1..1)
Entity : 'entity' name=ID ('extends' extends= ID )? '{' properties+=Property* '}'; Property: 'property' name=ID ':' type= ID (many?='[]')?; Results in containment relation “ List<Property> properties” Results in simple attribute “ String: name”

Example of Properties
Simple property:
name of SimpleType
“ Many” reference – containment
elements of Model
properties of Entity
Boolean property
many of Property
Model : (elements+=Type)*; Type: SimpleType | Entity; SimpleType: 'type' name=ID; Entity : 'entity' name=ID ('extends' extends= ID )? '{' properties+=Property* '}'; Property: 'property' name=ID ':' type= ID (many?='[]')?;

By default, rules results in a logical model with a tree of entities via containment
You can reference other elements via a reference rule
In textual languages a reference has to be by name
During linking, Xtext “dereferences” these by name-references
Why References?
Used in the logical model for references
XText also uses references for context assist
Reference Relations Entity : 'entity' name=ID ('extends' extends= [Entity] )? '{' properties+=Property* '}'; Results in containment relation “ List<Property> properties” Results in reference relation “ Entity extends”

Example of References
extends of Entity
type of Property
Model : (elements+=Type)*; Type: SimpleType | Entity; SimpleType: 'type' name=ID; Entity : 'entity' name=ID ('extends' extends=[Entity])? '{' properties+=Property* '}'; Property: 'property' name=ID ':' type=[Type] (many?='[]')?;

A Enum Rule is used to define a limited set of defined alternatives
It is mapped to an enumeration in the logical model
It is declared via the Enum keyword and contains Enum Literals
An Enum Literal has a token name and a string representation
It can be used just like any other rule
Properties will get the enumeration type
Enumeration Rules enum Color: RED=“red” | GREEN=“green” | BLUE=“blue” ; Shape: name=ID ( ‘color’ color=Color)? … ;

A native rule contains a string which is passed to ANTLR without further processing it.
It is typically used to define lexer rules that cannot be expressed using Xtext syntax
E.g. whitespace-aware lexer rules, such as define custom comment syntax
Native Rules

Importing an Existing Model
To import an model instead of generating it..
Simple change to the xtext grammar file:
Depending on the starting point a simple change to the MWE2 file:
Not so with the generated MWE2 file 
Easier to start with the wizard “XText Project from existing ECore Models”
grammar com.rcpcompany.mdd2010.DSL with org.eclipse.xtext.common.Terminals import "platform:/resource/com.rcpcompany.mdd2010.model/model/Travel.ecore" import "http://www.eclipse.org/emf/2002/Ecore" as ecore … fragment = org.eclipse.xtext.generator.ecore.EcoreGeneratorFragment { genModels = "platform:/resource/com.rcpcompany.mdd2010.model/model/Travel.genmodel" }

More Information XText and friends
“ Build your own textual DSL with Tools from the Eclipse Modeling Project”
http://www.eclipse.org/articles/article.php?file=Article-BuildYourOwnDSL/index.html
Older, slightly out-dated, article on how to create your first XText project
Documentation for XText
http:// www.eclipse.org/Xtext/documentation /

Exercise 1
Make sure you grammar is on proper EBNF form
Do you think you can make an interesting editor from the grammar?
Create an XText project if not already done
Convert your EBNF grammar into an XText model
What can you do to your grammar to make it more suitable for XText

ITU - MDD - XText

by Tonny Madsen on Nov 15, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 1

Statistics

ITU - MDD - XText — Presentation Transcript