Compiler Components and their Generators - LR Parsing

2,730 views
2,756 views

Published on

Presentation slides for lecture 13 of course IN4303 on Compiler Construction at TU Delft.

0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,730
On SlideShare
0
From Embeds
0
Number of Embeds
580
Actions
Shares
0
Downloads
149
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • non-intuitive classification\nnon-intuitive error messages\ncomplex parsing algorithms\n
  • \n
  • \n
  • \n
  • declarative disambiguation\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • interleaved with evaluation of the parser\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • not closed under composition\n
  • not closed under composition\n
  • not closed under composition\n
  • \n
  • overlapping morphem classes\nglobally reserved words\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • what did before the scanner, now does the parser\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • closed under composition\n
  • closed under composition\n
  • closed under composition\n
  • closed under composition\n
  • closed under composition\n
  • closed under composition\n
  • closed under composition\n
  • closed under composition\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Compiler Components and their Generators - LR Parsing

    1. 1. Compiler Components & GeneratorsTraditional Parsing AlgorithmsGuido Wachsmuth Delft Course IN4303 University of Technology Compiler Construction Challenge the future
    2. 2. Recap: Traditional Parsing Algorithmslessons learnedHow can we parse context-free languages effectively? • predictive parsing algorithms • LR parsing algorithmsWhich grammar classes are supported by these algorithms? • LL(k) grammars, LL(k) languages • LR(k) grammars, LR(k) languagesHow can we generate compiler tools from that? • implement automata • generate parse tables Traditional Parsing Algorithms 2
    3. 3. Overviewtoday’s lecture Lexical Analysis 3
    4. 4. Overviewtoday’s lectureefficient parsing algorithms • LR parse table generation • SLR & LALR parse tables Lexical Analysis 3
    5. 5. Overviewtoday’s lectureefficient parsing algorithms • LR parse table generation • SLR & LALR parse tablesplagues of traditional parsing algorithms • paradise lost • paradise regained Lexical Analysis 3
    6. 6. ILR parse tables Lexical Analysis 4
    7. 7. Recap: LR parsingexample stack input$ 7 * 3 + 7 * 3 $ Traditional Parsing Algorithms 5
    8. 8. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $ Traditional Parsing Algorithms 5
    9. 9. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $ Traditional Parsing Algorithms 5
    10. 10. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $ Traditional Parsing Algorithms 5
    11. 11. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $ Traditional Parsing Algorithms 5
    12. 12. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $ Traditional Parsing Algorithms 5
    13. 13. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $ Traditional Parsing Algorithms 5
    14. 14. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $ Traditional Parsing Algorithms 5
    15. 15. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $ Traditional Parsing Algorithms 5
    16. 16. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $ Traditional Parsing Algorithms 5
    17. 17. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $ Traditional Parsing Algorithms 5
    18. 18. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $ Traditional Parsing Algorithms 5
    19. 19. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $ Traditional Parsing Algorithms 5
    20. 20. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $$ E + E * E $ Traditional Parsing Algorithms 5
    21. 21. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $$ E + E * E $$ E + E $ Traditional Parsing Algorithms 5
    22. 22. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $$ E + E * E $$ E + E $$ E $ Traditional Parsing Algorithms 5
    23. 23. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $$ E + E * E $$ E + E $$ E $ Traditional Parsing Algorithms 5
    24. 24. Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $$ E + E * E $$ E + E $$ E $$ S Traditional Parsing Algorithms 5
    25. 25. Recap: LR parsingparse tablerows T1 ... N1 ... • states of a DFA 1 s3 2 a g5columns 3 r1 • topmost stack symbol, Σ, N 4 r2entries 5 • reduce, rule number 6 g1 • shift, goto state 7 s1 • goto state 8 • accept ... Traditional Parsing Algorithms 6
    26. 26. LR(0) parse tablesitems, closure & goto S→x S→(L) L→S L→L,S Traditional Parsing Algorithms 7
    27. 27. LR(0) parse tablesitems, closure & gotoS’ → . S $ S→x S→(L) L→S L→L,S Traditional Parsing Algorithms 7
    28. 28. LR(0) parse tablesitems, closure & goto itemS’ → . S $ S→x S→(L) L→S L→L,S Traditional Parsing Algorithms 7
    29. 29. LR(0) parse tablesitems, closure & goto item closure • for every item A → α . X βS’ → . S $ • for every rule X → γ • add item X → . γ S→x S→(L) L→S L→L,S Traditional Parsing Algorithms 7
    30. 30. LR(0) parse tablesitems, closure & goto closure • for every item A → α . X βS’ → . S $S→.x • for every rule X → γS→.(L) • add item X → . γ S→x S→(L) L→S L→L,S Traditional Parsing Algorithms 7
    31. 31. LR(0) parse tablesitems, closure & gotoS’ → . S $S→.xS→.(L) S→x S→(L) L→S L→L,S Traditional Parsing Algorithms 7
    32. 32. LR(0) parse tablesitems, closure & gotoS’ → . S $S→.xS→.(L) S S→x S→(L) L→S L→L,SS’ → S . $ Traditional Parsing Algorithms 7
    33. 33. LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.xS→.(L) S S→x S→(L) L→S L→L,SS’ → S . $ Traditional Parsing Algorithms 7
    34. 34. LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.xS→.(L) ( S→(.L) S S→x S→(L) L→S L→L,SS’ → S . $ Traditional Parsing Algorithms 7
    35. 35. LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.xS→.(L) ( S→(.L) L→.S L→.L,S S→.x S→x S S→.(L) S→(L) L→S L→L,SS’ → S . $ Traditional Parsing Algorithms 7
    36. 36. LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.x xS→.(L) ( S→(.L) L→.S L→.L,S S→.x S→x S S→.(L) S→(L) L→S L→L,SS’ → S . $ Traditional Parsing Algorithms 7
    37. 37. LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.x xS→.(L) ( S→(.L) L→.S L→.L,S ( S→.x S S→x S→.(L) S→(L) L→S L→L,SS’ → S . $ Traditional Parsing Algorithms 7
    38. 38. LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.x xS→.(L) ( S→(.L) L→.S L→.L,S ( S→.x S S→x S→.(L) S→(L) S L→S L→L,SS’ → S . $ L→S. Traditional Parsing Algorithms 7
    39. 39. LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.x xS→.(L) ( S→(.L) L→.S L→.L,S ( S→.x S L S→(L.) S→x S→.(L) L→L.,S S→(L) S L→S L→L,SS’ → S . $ L→S. Traditional Parsing Algorithms 7
    40. 40. LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.x xS→.(L) ( S→(.L) L→.S L→.L,S ( S→.x S L S→(L.) S→x S→.(L) L→L.,S S→(L) S ) L→S L→L,SS’ → S . $ L→S. S→(L). Traditional Parsing Algorithms 7
    41. 41. LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.x xS→.(L) ( L→L,.S S→(.L) L→.S , L→.L,S ( S→.x S L S→(L.) S→x S→.(L) L→L.,S S→(L) S ) L→S L→L,SS’ → S . $ L→S. S→(L). Traditional Parsing Algorithms 7
    42. 42. LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $ L→L,.SS→.x x S→.xS→.(L) ( S→.(L) S→(.L) L→.S , L→.L,S ( S→.x S L S→(L.) S→x S→.(L) L→L.,S S→(L) S ) L→S L→L,SS’ → S . $ L→S. S→(L). Traditional Parsing Algorithms 7
    43. 43. LR(0) parse tablesitems, closure & goto x x S→x.S’ → . S $ L→L,.SS→.x x S→.xS→.(L) ( S→.(L) S→(.L) L→.S , L→.L,S ( S→.x S L S→(L.) S→x S→.(L) L→L.,S S→(L) S ) L→S L→L,SS’ → S . $ L→S. S→(L). Traditional Parsing Algorithms 7
    44. 44. LR(0) parse tablesitems, closure & goto x x S→x.S’ → . S $ L→L,.SS→.x x S→.xS→.(L) ( ( S→.(L) S→(.L) L→.S , L→.L,S ( S→.x S L S→(L.) S→x S→.(L) L→L.,S S→(L) S ) L→S L→L,SS’ → S . $ L→S. S→(L). Traditional Parsing Algorithms 7
    45. 45. LR(0) parse tablesitems, closure & goto L→L,S. S x x S→x.S’ → . S $ L→L,.SS→.x x S→.xS→.(L) ( ( S→.(L) S→(.L) L→.S , L→.L,S ( S→.x S L S→(L.) S→x S→.(L) L→L.,S S→(L) S ) L→S L→L,SS’ → S . $ L→S. S→(L). Traditional Parsing Algorithms 7
    46. 46. LR(0) parse tablesitems, closure & goto L→L,S.9 S x x S→x. 2S’ → . S $ 1 L→L,.S8S→.x x S→.xS→.(L) ( ( S→.(L) S→(.L)3 L→.S , L→.L,S ( S→.x S L S→(L.)5 S→x S→.(L) L→L.,S S→(L) S ) L→S L→L,SS’ → S . $ 4 L→S. 6 S→(L).7 Traditional Parsing Algorithms 7
    47. 47. LR(0) parse tablesresult ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 7 r2 r2 r2 r2 r2 L→L,S 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    48. 48. LR(0) parse tablesresult ( x , x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    49. 49. LR(0) parse tablesresult x , x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    50. 50. LR(0) parse tablesresult , x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) 2 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    51. 51. LR(0) parse tablesresult , x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    52. 52. LR(0) parse tablesresult , x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) 6 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    53. 53. LR(0) parse tablesresult , x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    54. 54. LR(0) parse tablesresult , x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) 5 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    55. 55. LR(0) parse tablesresult x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 8 S→x 6 r3 r3 r3 r3 r3 S→(L) 5 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    56. 56. LR(0) parse tablesresult ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 2 5 s7 s8 8 S→x 6 r3 r3 r3 r3 r3 S→(L) 5 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    57. 57. LR(0) parse tablesresult ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 8 S→x 6 r3 r3 r3 r3 r3 S→(L) 5 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    58. 58. LR(0) parse tablesresult ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 9 5 s7 s8 8 S→x 6 r3 r3 r3 r3 r3 S→(L) 5 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    59. 59. LR(0) parse tablesresult ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    60. 60. LR(0) parse tablesresult ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) 5 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    61. 61. LR(0) parse tablesresult $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 7 S→x 6 r3 r3 r3 r3 r3 S→(L) 5 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    62. 62. LR(0) parse tablesresult $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    63. 63. LR(0) parse tablesresult $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 4 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
    64. 64. SLR parse tablesshift-reduce conflicts E S→E.$S→.E$E→.T+EE→.T T E→T.+ET→.x E→T. T + x E→T+.E E→.T+E E→T+E E→.T E→TT→x. x T→.x E E→T+E. T→x Traditional Parsing Algorithms 9
    65. 65. SLR parse tablesshift-reduce conflicts E S→E.$ 2S→.E$E→.T+EE→.T T E→T.+ET→.x 1 E→T. 3 T + x E→T+.E E→.T+E E→T+E E→.T E→TT→x. x T→.x E E→T+E. 6 5 4 T→x Traditional Parsing Algorithms 9
    66. 66. SLR parse tablesshift-reduce conflicts x + $ E T 1 s5 g2 g3 E S→E.$ 2 2 aS→.E$E→.T+E 3 r2 ? r2E→.T T 4 s5 g6 g3 E→T.+ET→.x 1 5 r3 r3 r3 E→T. 3 6 r1 r1 r1 T + x E→T+.E E→.T+E E→T+E E→.T E→TT→x. x T→.x E E→T+E. 6 5 4 T→x Traditional Parsing Algorithms 9
    67. 67. SLR parse tablesshift-reduce conflicts x + $ E T 1 s5 g2 g3 E S→E.$ 2 2 aS→.E$E→.T+E 3 s4 r2E→.T T 4 s5 g6 g3 E→T.+ET→.x 1 5 r3 r3 E→T. 3 6 r1 T + x E→T+.E E→.T+E E→T+E E→.T E→TT→x. x T→.x E E→T+E. 6 5 4 T→x Traditional Parsing Algorithms 9
    68. 68. LR(1) parse tableslook-ahead E S→E.$ ?S→.E$ ?E→.T+E $E→.T $ T E→T.+E $T→.x +$ E→T. $ T + x E→T+.E $ E→.T+E $ E→T+E E→.T $ E→TT→x. +$ x T→.x +$ E E→T+E. $ T→x Traditional Parsing Algorithms 10
    69. 69. LR(1) parse tableslook-ahead x + $ E T 1 s5 g2 g3 E S→E.$ ? 2 aS→.E$ ?E→.T+E $ 3 s4 r2E→.T $ T 4 s5 g6 g3 E→T.+E $T→.x +$ 5 r3 r3 E→T. $ 6 r1 T + x E→T+.E $ E→.T+E $ E→T+E E→.T $ E→TT→x. +$ x T→.x +$ E E→T+E. $ T→x Traditional Parsing Algorithms 10
    70. 70. LALR(1) parse tablesstate space reductionunify states • with same items • and same outgoing transitions • but different look-ahead setsmight introduce new conflicts Traditional Parsing Algorithms 11
    71. 71. coffee break Traditional Parsing Algorithms 12
    72. 72. IIparadise lost & regained Lexical Analysis 13
    73. 73. PARADISE LOST
    74. 74. EFFICIENCY
    75. 75. THE FIRST PLAGUE WEREGRAMMAR CLASSES
    76. 76. context-free grammars
    77. 77. context-free grammars LL(0)
    78. 78. context-free grammars LL(1) LL(0)
    79. 79. context-free grammars LL(k) LL(1) LL(0)
    80. 80. context-free grammars LL(k) LL(1) LL(0) LR(0)
    81. 81. context-free grammars LL(k) LL(1) LR(1) LL(0) LR(0)
    82. 82. context-free grammars LL(k) LR(k) LL(1) LR(1) LL(0) LR(0)
    83. 83. context-free grammars LL(k) LR(k) LL(1) LR(1) SLR LL(0) LR(0)
    84. 84. context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
    85. 85. Exp “+” Exp ➝ Exp Term (“+” Term)* ➝ ExpExp “*” Exp ➝ Exp Fact (“*” Fact)* ➝ Term Num ➝ Exp Num ➝ Fact paradise grammar classes
    86. 86. Exp * Exp * Exp Term Term TermExp Exp Exp Fact Fact Fact3 + 7 + 21 3 + 7 + 21 paradise grammar classes
    87. 87. THE SECOND PLAGUE WASDISAMBIGUATION
    88. 88. Exp Exp Exp ExpExp Exp Exp Exp Exp Exp3 * 7 + 21 3 * 7 + 21
    89. 89. precedence operators associativity 1 ( ), [ ] non-associative 2 new non-associative 3 . left-associative 4 ++, -- non-associative 5 -, +, !, ~, ++, --, (type) right-associative 6 *, /, % left-associative 7 +, - left-associative … … … text books
    90. 90. context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
    91. 91. context-free grammars unambigous LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
    92. 92. Exp “+” Term ➝ Exp Term ➝ Exp Term “*” Fact ➝ Term Fact ➝ Term Num ➝ Factgrammar classes
    93. 93. FALSE PROPHETS
    94. 94. L = {ab, a} L = {ab, a}“a” “b” ➝ A “a” “b” / “a” ➝ A “a” ➝Aparadise PEGs
    95. 95. L = {ab, a} L = {a} “a” ➝ A “a” /“a” “b” ➝ A “a” “b” ➝ Aparadise PEGs
    96. 96. if c1 then if c2 then s1 else s2 dangling else
    97. 97. “if ” E “then” S “else” S / “if ” E “then” S /“if ” E “then” S ➝S “if ” E “then” S “else” S ➝ S PEGs
    98. 98. THE THIRD PLAGUE WASLEXICAL SYNTAX
    99. 99. morphology & syntax
    100. 100. LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)limited look-ahead
    101. 101. scanners
    102. 102. 3 * 7 + 21
    103. 103. 3 * 7 + 21
    104. 104. parsers
    105. 105. 3 * 7 + 21
    106. 106. Exp ExpExp Exp Exp3 * 7 + 21
    107. 107. x = 1. * .10y : array [ 1 .. 10 ] of integer
    108. 108. x = 1. * .10y : array [ 1 .. 10 ] of integer
    109. 109. x = 1. * .10y : array [ 1. .10 ] of integer
    110. 110. THE FOURTH PLAGUE WASTREE CONSTRUCTION
    111. 111. Exp Add Exp AddExp Exp Exp Const Const Const3 + 7 + 21 3 7 21 paradise
    112. 112. Exp * Add * AddTerm Term Term Const Const ConstFact Fact Fact 3 7 21 3 + 7 + 21 grammar classes
    113. 113. expr: INTEGER { $$ = con($1); }| expr + expr { $$ = opr(+, 2, $1, $3);}| expr * expr { $$ = opr(*, 2, $1, $3);}; semantic actions
    114. 114. THE FIFTH PLAGUE WAS EVOLUTION
    115. 115. Exp “+” Term ➝ ExpExp “+” Exp ➝ Exp Term ➝ ExpExp “*” Exp ➝ Exp Term “*” Fact ➝ Term Num ➝ Exp Fact ➝ Term Num ➝ Fact paradise grammar classes
    116. 116. CExp “+” Term ➝ CExp Term ➝ CExpExp “+” Exp ➝ Exp Term “*” Fact ➝ TermExp “*” Exp ➝ Exp Fact ➝ Term Num ➝ Exp Num ➝ FactExp “=” Exp ➝ Exp Exp “=” CExp ➝ ExpExp “<” Exp ➝ Exp Exp “<” CExp ➝ ExpExp “>” Exp ➝ Exp Exp “>” CExp ➝ Exp CExp ➝ Exp paradise grammar classes
    117. 117. THE SIXTH PLAGUE WASCOMPOSITION
    118. 118. parsers
    119. 119. context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
    120. 120. context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
    121. 121. context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
    122. 122. context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
    123. 123. scanners
    124. 124. public boolean authenticate(String user, String pw) { SQL stm = <| SELECT id FROM Users WHERE name = ${user} AND password = ${pw} |>; return executeQuery(stm).size() != 0;}
    125. 125. THE SEVENTH PLAGUE WASRESTRICTION TO PARSERS
    126. 126. PRETTY PRINTERS
    127. 127. SENTENCEGENERATORS
    128. 128. AST ACCESS
    129. 129. IDE SUPPORT
    130. 130. PARADISEREGAINED
    131. 131. GENERALISEDPARSING
    132. 132. context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
    133. 133. context-free grammars
    134. 134. context-free syntaxExp “+” Exp ➝ ExpExp “*” Exp ➝ Exp Exp "+" Exp -> Exp Num ➝ Exp Exp "*" Exp -> Exp NUM -> Exp paradise SDF
    135. 135. DECLARATIVEDISAMBIGUATION
    136. 136. context-free grammars unambigous LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
    137. 137. context-free grammars unambigous
    138. 138. context-free grammars
    139. 139. precedence operators associativity 1 ( ), [ ] non-associative 2 new non-associative 3 . left-associative 4 ++, -- non-associative 5 -, +, !, ~, ++, --, (type) right-associative 6 *, /, % left-associative 7 +, - left-associative … … … text books
    140. 140. context-free priorities Exp "*" Exp -> Exp {left}> Exp "+" Exp -> Exp {left}
    141. 141. context-free syntax "if" E "then" S -> S {prefer} "if" E "then" S "else" S -> S
    142. 142. SCANNERLESSPARSING
    143. 143. morphology & syntax
    144. 144. lexical syntax [0-9]+ -> NUM [ tn] -> LAYOUT "//" ~[n]* [n] -> LAYOUT
    145. 145. parser
    146. 146. parser
    147. 147. DECLARATIVETREE CONSTRUCTION
    148. 148. Exp Add Exp AddExp Exp Exp Const Const Const3 + 7 + 21 3 7 21 paradise
    149. 149. context-free syntax Exp "+" Exp -> Exp {cons("Add")} Exp "*" Exp -> Exp {cons("Mul")} NUM -> Exp {cons("Const")} SDF
    150. 150. Add Add( Add( Add Const("3"), Const("7")Const Const Const ), Const("21") 3 7 21 ) paradise SDF
    151. 151. SEAMLESSEVOLUTION
    152. 152. context-free syntax Exp "+" Exp -> Exp {cons("Add")} Exp "*" Exp -> Exp {cons("Mul")} NUM -> Exp {cons("Const")} Exp "=" Exp -> Exp {cons("Eq")} Exp ">" Exp -> Exp {cons("Gt")} Exp "<" Exp -> Exp {cons("Lt")}
    153. 153. MODULARCOMPOSITION
    154. 154. context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
    155. 155. context-free grammars
    156. 156. public boolean authenticate(String user, String pw) { SQL stm = <| SELECT id FROM Users WHERE name = ${user} AND password = ${pw} |>; return executeQuery(stm).size() != 0;}
    157. 157. module Java-SQL imports Java SQL exports context-free syntax "<|" Query "|>" -> Exp {cons("ToSQL")} "${" Exp "}" -> SqlExp {cons("FromSQL")}
    158. 158. BEYONDPARSERS
    159. 159. PRETTY PRINTERS
    160. 160. SENTENCEGENERATORS
    161. 161. AST ACCESS
    162. 162. IDE SUPPORT
    163. 163. IIIsummary Lexical Analysis 88
    164. 164. Summarylessons learned Traditional Parsing Algorithms 89
    165. 165. Summarylessons learnedHow can we generate LR parse tables? • items, closure, goto Traditional Parsing Algorithms 89
    166. 166. Summarylessons learnedHow can we generate LR parse tables? • items, closure, gotoHow can we improve LR(0) parse table generation? • SLR: consider FOLLOW sets to avoid shift-reduce conflicts • LR(1): consider look-ahead in states • LALR(1): unify LR(1) states to reduce state space Traditional Parsing Algorithms 89
    167. 167. Summarylessons learnedHow can we generate LR parse tables? • items, closure, gotoHow can we improve LR(0) parse table generation? • SLR: consider FOLLOW sets to avoid shift-reduce conflicts • LR(1): consider look-ahead in states • LALR(1): unify LR(1) states to reduce state spaceWhy are efficient parsing algorithms problematic? • not longer pure, declarative, beautiful • paradise lost: seven plagues • paradise regained: scannerless generalised parsing Traditional Parsing Algorithms 89
    168. 168. Summarylessons learnedHow can we generate LR parse tables? • items, closure, gotoHow can we improve LR(0) parse table generation? • SLR: consider FOLLOW sets to avoid shift-reduce conflicts • LR(1): consider look-ahead in states • LALR(1): unify LR(1) states to reduce state spaceWhy are efficient parsing algorithms problematic? • not longer pure, declarative, beautiful • paradise lost: seven plagues • paradise regained: scannerless generalised parsing Traditional Parsing Algorithms 89
    169. 169. Literaturelearn more Traditional Parsing Algorithms 90
    170. 170. Literaturelearn moresyntactical analysis Andrew W. Appel, Jens Palsberg: Modern Compiler Implementation in Java, 2nd edition. 2002 Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman, Monica S. Lam: Compilers: Principles, Techniques, and Tools, 2nd edition. 2006 Traditional Parsing Algorithms 90
    171. 171. Literaturelearn moresyntactical analysis Andrew W. Appel, Jens Palsberg: Modern Compiler Implementation in Java, 2nd edition. 2002 Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman, Monica S. Lam: Compilers: Principles, Techniques, and Tools, 2nd edition. 2006generalised parsing Eelco Visser: Syntax Definition for Language Prototyping. PhD thesis 1997 M.G.J. van den Brand, J. Scheerder, J.J. Vinju, and E. Visser: Disambiguation Filters for Scannerless Generalized LR Parsers. CC 2002 Lennart C. L. Kats, Eelco Visser, Guido Wachsmuth: Pure and Declarative Syntax Definition - Paradise Lost and Regained. SPLASH 2010 Traditional Parsing Algorithms 90
    172. 172. Outlookcoming nextlectures • Lecture 14: Beyond grammarwareQuestion & Answer Jan 5 • 10 questions, submit & voteLab Dec 10 • support expressions • support statements • support ordinary classes & fields Traditional Parsing Algorithms 91
    173. 173. questions Lexical Analysis 92
    174. 174. credits Lexical Analysis 93
    175. 175. PicturescopyrightsSlide 1: Book Scanner by Ben Woosley, some rights reservedSlide 12: West Cornwall Pasty Co. by Dominica Williamson, some rights reservedSlide 91: Ostsee by Mario Thiel, some rights reservedSlide 92: Questions by Oberazzi, some rights reservedSlide 93: Too Much Credit by Andres Rueda, some rights reservedSlides 14-87: Pure and Declarative Syntax Definition: Paradise Lost and Regained, some rights reserved Traditional Parsing Algorithms 94

    ×