Mastering Grammars with PetitParser

2,709 views

Published on

PetitParser is a dynamic parser framework combining ideas from scannerless parsing, parser combinators, parsing expression grammars and packrat parsers. In this hands-on session we learn how to build simple parsers and how to model, test, compose and reuse complex grammars. Additionally, we will look at some tools and the reflective facilities provided by the PetitParser framework. Basic knowledge of the Smalltalk programming language is a requirement. Bring your laptop to reproduce the examples and solve some simple tasks.

Published in: Technology, Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,709
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
23
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Mastering Grammars with PetitParser

  1. 1. Mastering Grammars with Lukas Renggli
  2. 2. http://dsal.cl/~jfabry/pp/
  3. 3. Agenda1. PetitParser in a Nutshell2. Combinatorial Parsing3. Complex Grammars4. Advanced Use
  4. 4. in a Nutshell
  5. 5. ID  ::=  letter  {  letter  |  digit  }  ; a..z a..z 0..9
  6. 6. ID  ::=  letter  {  letter  |  digit  }  ; sequence letter many choice letter digit
  7. 7. id  :=  #letter  asParser  ,  (#letter  asParser  /  #digit  asParser)  star sequence letter many choice letter digit
  8. 8. id  parse:  yeah.   "  -­‐-­‐>  #($y  #($e  $a  $h))  "id  parse:  f12.   "  -­‐-­‐>  #($f  #($1  $2))  "id  parse:  123.   "  -­‐-­‐>  letter  expected  at  0  "
  9. 9. Example 1‣ Test the identifier parser‣ Inspect the parser object‣ Create and test an integer parser
  10. 10. Combinatorial Parsing
  11. 11. Parser Terminals $a  asParser letter ‘a’ abc  asParser string ‘abc’These factory methods are defined in PPPredicateObjectParser  class #any  asParser any character #digit  asParser the digits 0..9 #letter  asParser the letters a..z and A..Z nil  asParser the empty parser
  12. 12. Parser Operatorsp1  ,  p2   sequencep1  /  p2   ordered choicep  star   zero-or-more (0..*)p  plus   one-or-more (1..*)p  optional   zero-or-one (0..1) see the operationsp1  separatedBy:  p2 protocols in PPParserp1  delimitedBy:  p2 for more operators
  13. 13. Parser Predicatesp  and   conjunction (non-consuming look-ahead)p  not   negation (non-consuming look-ahead)p  end   end of input
  14. 14. Parser Actions p  ==>  [  :arg  |      ] transformation   p  flatten   create string p  token   create token p  trim trim whitespacessee the operations-mapping protocol in PPParser for more actions
  15. 15. term          ::=  prod  "+"  term                        |  prod  ;prod          ::=  prim  "*"  prod                      |  prim  ;prim          ::=  "("  term  ")"                      |  number  ;number      ::=  "0"  ..  "9"  ;
  16. 16. number  :=    #digit  asParser  plus  flatten  trim   ==>  [  :string  |  string  asNumber  ].term  :=  PPUnresolvedParser  new.prod  :=  PPUnresolvedParser  new.prim  :=  PPUnresolvedParser  new.  term  def:  (prod  ,  $+  asParser  trim  ,  term     ==>  [  :nodes  |  nodes  first  +  nodes  last  ])      /  prod.prod  def:  (prim  ,  $*  asParser  trim  ,  prod   ==>  [  :nodes  |  nodes  first  *  nodes  last  ])      /  prim.prim  def:  ($(  asParser  trim  ,  term  ,  $)  asParser  trim   ==>  [  :nodes  |  nodes  second  ])      /  number.start  :=  term  end.
  17. 17. start  parse:  1  +  2  *  3.           "  -­‐-­‐>  7  "start  parse:  (1  +  2)  *  3.       "  -­‐-­‐>  9  "
  18. 18. Example 2‣ Add support for negative numbers‣ Add support for floating point numbers‣ Add support for subtraction & division
  19. 19. Complex Grammars
  20. 20. PetitParser Scripts
  21. 21. toqu ick wr itePetitParser Scripts embe d in Smal to ltalk
  22. 22. to qu ick d iffic ult wr ite to reu se PetitParser Scripts embe messy hard d in to if large Smalto tes t ltalk
  23. 23. PPCompositeParserstart
  24. 24. PPCompositeParserstart ExpressionGrammarstarttermprod...
  25. 25. One Method per Production PPCompositeParser start ExpressionGrammar start start term      ^  term  end prod ...
  26. 26. One Instance Variable per Production PPCompositeParser start ExpressionGrammar start term term      ^  (prod  ,  $+  asParser  t prod        /  prod ...
  27. 27. Refer to Productions by Inst-Var Reference PPCompositeParser start ExpressionGrammar start term prod prod      ^  (prim  ,  $*  asParser  t ...        /  prim
  28. 28. res t is PPCompositeParser mag icstart ExpressionGrammarstarttermprod...
  29. 29. ExpressionGrammarstarttermprod...ExpressionEvaluatortermprodnumber...
  30. 30. Example 3‣ Implement an expression grammar‣ Implement an expression evaluator‣ Implement an expression pretty printer
  31. 31. Advanced Use
  32. 32. does not justwork on Strings
  33. 33. Matchingp  matches:  abc.p  matchesIn:  abc.  p  matchesIn:  abc  do:  [  :each  |      ].p  matchingRangesIn:  abc.p  matchingRangesIn:  abc  do:  [  :interval  |      ].  
  34. 34. World ➔ Tools ➔ PetitParser GUI PPBrowser  open
  35. 35. Reflectionp  allParser.p  allParserDo:  [  :each  |      ].p  firstSet.  p  followSet.p  cycleSet.
  36. 36. Transformationsp  replace:  p1  with:  p2.p  transform:  [  :parser  |      ]. Like #collect: on Collection, but transforms the whole grammar graph.
  37. 37. Pattern Searching A placeholder matching any parserfound  :=  PPSearcher  new   matches:  PPPattern  any  star  plus     do:  [  :parser  :answer  |  parser  ];   execute:  p  initialAnswer:  nil Grammar to be searched
  38. 38. Pattern RewritingSame pattern used in search and replace. pattern  :=  PPPattern  any. rewritten  :=  PPRewriter  new   replace:  pattern  star  plus   with:  pattern  star;   execute:  p Grammar to be rewritten
  39. 39. Grammar Optimization fast  :=  slow  optimize Many behavior preserving rewrite rules applied for you.
  40. 40. 3000000 2000000chars/sec 1000000 0 LALR PetitParser Hand-Written Old VM Cog VM
  41. 41. www.lukas-renggli.ch

×