The Xtext Grammar Language

3,155 views

Published on

Slides of a talk held at XtextCON 2014 (Kiel)

Published in: Software
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,155
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
41
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

The Xtext Grammar Language

  1. 1. Xtext Grammar Language Jan Köhnlein 2014, Kiel
  2. 2. grammar org.xtextcon.Statemachine with org.eclipse.xtext.common.Terminals ! generate statemachine "http://www.xtextcon.org/Statemachine" ! Statemachine: name=STRING elements+=Element*; ! Element: Event | State; Event: name=ID description=STRING?; ! State: 'state' name=ID transitions+=Transition*; ! Transition: event=[Event] '=>' state=[State];
  3. 3. Precedences Action, Assignment, Keyword, 
 RuleCall, Parenthesized Cardinalities *, +, ?
 Predicates =>, -> Group <blank> Unordered Group & Alternative | AssignmentFirst: name=ID?; // (name=ID)?; ! CardinalityFirst: 'a' 'b'?; // 'a' ('b'?); ! PredicateFirst: =>'a' 'b'; // (=>'a') 'b'; ! GroupFirst: 'a' | 'b' 'c'; // 'a' | ('b' 'c');
  4. 4. Syntactic Aspects
  5. 5. grammar org.xtextcon.Statemachine with org.eclipse.xtext.common.Terminals ! generate statemachine "http://www.xtextcon.org/Statemachine" ! Statemachine: name=STRING ('events' events+=Event+)? states+=State*; ! Event: name=ID description=STRING?; ! State: 'state' name=ID transitions+=Transition*; ! Transition: event=[Event|ID] '=>' state=[State|ID];
  6. 6. Lexing
  7. 7. Lexer Splits document text into tokens Works before and independent of the parser Token matching First match wins Precedence All keywords Local terminal rules Terminal rules from super grammar As last resort you can define a custom lexer
  8. 8. Lexing // Examplen ’Lamp’ n events button_pressed n state on nt button_pressed => off n state off nt button_pressed => light SL_COMMENT STRING WS events WS ID WS state ID WS ID WS => WS ID WS state ID WS ID WS => WS ID events
 state
 =>
 ID
 STRING
 ML_COMMENT
 SL_COMMENT WS ANY_OTHER Tokens Token StreamChar Stream
  9. 9. Terminal Rules grammar org.eclipse.xtext.common.Terminals hidden(WS, ML_COMMENT, SL_COMMENT) import "http://www.eclipse.org/emf/2002/Ecore" as ecore ! terminal ID : '^'?('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*; terminal INT returns ecore::EInt: ('0'..'9')+; terminal STRING : '"' ( '' ('b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'') | !(''|'"') )* '"' | "'" ( '' ('b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'') | !(''|"'") )* "'"; terminal ML_COMMENT : '/*' -> '*/'; terminal SL_COMMENT : '//' !('n'|'r')* ('r'? 'n')?; ! terminal WS : (' '|'t'|'r'|'n')+; ! terminal ANY_OTHER : .;
  10. 10. Terminals in the Parser
  11. 11. Hidden Terminals Are ignored by the parser inside the respective scope Can be defined on grammar or rule level (inherited)
 
 grammar org.eclipse.xtext.common.Terminals 
 hidden(WS, ML_COMMENT, SL_COMMENT) or on parser rules
 
 QualifiedName hidden(): // strictly no whitespace ID ('.' ID)*;
  12. 12. Datatype Rules Return a value (instead of an EObject) Are processed by the parser Superior alternative to terminal rules Examples
 
 Double returns ecore::EDouble hidden():
 '-'? INT '.' INT ('e'|'E' '-'? INT)?;
 
 QualifiedName: // returns ecore::EString (default)
 ID ('.' ID)*;
 
 ValidID:
 ID | 'keyword0' | 'keyword1';
  13. 13. Transforms textual representation to value and back For terminal and datatype rules e.g. strips quotes (STRINGValueConverter)
 remove leading ^ (IDValueConverter) EMF defaults are often sufficient (EFactory) Value Converter class MyValueConverterService extends AbstractDeclarativeValueConverterService { @Inject MyValueConverter myValueConverter; ! @ValueConverter(rule = "MY_TERMINAL") def MY_TERMINAL() { return myValueConverter } }
  14. 14. Ambiguities
  15. 15. Example Expression: If; If: Variable | 'if' condition=Expression 'then' then=Expression ('else' else=Expression)?; Variable: name=ID; warning(200): ...: Decision can match input such as "'else'" using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): ...: Decision can match input such as "'else'" using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input if cond1 then if cond2 then foo else // dangling bar
  16. 16. Ambiguity Analysis In the workflow, add ! AntlrWorks www.antlr3.org/works/ fragment = DebugAntlrGeneratorFragment { options = auto-inject {} }
  17. 17. Ambiguity Resolution Add keywords Use syntactic predicates:
 “If the predicated element matches in the current decision, decide for it.” plain
 (=>'else' else=Expression)?; 
 =>('else' else=Expression)?; // not so good first-set
 ->('else' else=Expression)?;
  18. 18. Correspondence to Ecore
  19. 19. grammar org.xtextcon.Statemachine with org.eclipse.xtext.common.Terminals ! generate statemachine "http://www.xtextcon.org/Statemachine" ! Statemachine returns Statemachine: name=STRING ('events' events+=Event+)? states+=State*; ! Event returns Event: name=ID description=STRING?; ! State returns State: 'state' name=ID transitions+=Transition*; ! Transition returns Transition: event=[Event|ID] '=>' state=[State|ID]; statemachine name : String Statemachine name : String Transition name : String State name : String description: String Event * * *states events transitions event1 1 state
  20. 20. Supertypes ... ! Element: Event | State; ! Event returns Event: name=ID description=STRING?; ! State returns State: 'state' name=ID transitions+=Transition*; ! ... description: String EventState name : String Element name : String Transition Common features are promoted to the super type Dispatcher rule Element needs not to be called A rule can return a subtype of the specified return type
  21. 21. Imported EPackage Workflow { bean = StandaloneSetup { ... registerGeneratedEPackage =
 “org.xtextcon.statemachine.StatemachinePackage” registerGenModelFile = 
 “platform:/resource/<path-to>/Statemachine.genmodel" } //generate statemachine "http://www.xtextcon.org/Statemachine" import "http://www.xtextcon.org/Statemachine"
  22. 22. AST Creation
  23. 23. EObject Creation The current pointer Points to the EObject returned by a parser rule call Is set to null when entering the rule On the first assignment the EObject is created and assigned to current Further assignments go to current
  24. 24. EObject Creation Type : (Class | DataType) 'is' visibility=('public' |'private'); DataType : 'datatype' name=ID; Class : 'class' name=ID; datatype A is public Type current = null; current = new DataType(); current.setName(“A”); current.setVisibility(“public”); return current;
  25. 25. Actions Alternative without assignment -> no returned object
 
 
 Actions make sure an EObject is created Specified type must be compatible with return type Created element is assigned to current BooleanLiteral: value?='true' | 'false'; BooleanLiteral returns Expression: {BooleanLiteral} (value?=‘true’ | ‘false'); The rule 'BooleanLiteral' may be consumed without object instantiation. Add an action to ensure object creation, e.g. '{BooleanLiteral}'.
  26. 26. Left Recursion Expression :   Expression '+' Expression |   Number; ! Number : value = INT; The rule 'Expression' is left recursive.
  27. 27. Left Factoring Expression: {Addition} left=Number ('+' right=Number)*; Number: value = INT; 1 // model Addition { left = Number { value = 1 } }
  28. 28. Assigned Action Addition returns Expression: Number ({Sum.left = current} '+' right=Number)*; Number: value = INT; 1 Expression current = null; current = new Number(); current.setValue(1); return current; // model Number { value = 1 }
  29. 29. Assigned Action II Addition returns Expression: Number ({Sum.left = current} '+' right=Number)*; Number: value = INT; 1 + 2 Expression current = null; current = new Number(); current.setValue(1); Expression temp = new Sum(); temp.setLeft(current); current = temp; … current.right = <second Number>; return current; // model Addition { left = Number { value = 1 } right = Number { value = 2 } }
  30. 30. Assigned Action III Creates a new element makes it the parent of current and sets current to it Needed normalizing left-factored grammars infix operators
  31. 31. Best Practices
  32. 32. Best Practices Use nsURIs for imported EPackages Prefer datatype rules to terminal rules Use syntactic predicates instead of backtracking Be sloppy in the grammar but strict in validation

×