Successfully reported this slideshow.
Upcoming SlideShare
×

# LR Parsing

9,720 views

Published on

Lecture for Compiler Construction course at TU Delft

Published in: Software
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Sex in your area is here: ♥♥♥ http://bit.ly/2F7hN3u ♥♥♥

Are you sure you want to  Yes  No

Are you sure you want to  Yes  No

### LR Parsing

1. 1. Challenge the future Delft University of Technology Course IN4303 Compiler Construction Eduardo Souza, Guido Wachsmuth, Eelco Visser LR Parsing Traditional Parsing Algorithms
2. 2. lessons learned LR Parsing Recap: Traditional Parsing Algorithms How can we parse context-free languages effectively? • predictive parsing Which grammar classes are supported by these algorithms? • LL(k) grammars, LL(k) languages How can we generate compiler tools from that? • implement automaton • generate parse tables What are other techniques for implementing top-down parsers? • Parser Combinators • PEGs • ALL(*) 2
3. 3. today’s lecture Lexical Analysis Overview 3
4. 4. today’s lecture Lexical Analysis Overview efficient parsing algorithms • LR parsing • LR parse table generation • SLR & LALR parse tables • Generalized LR parsing • Scannerless Generalized LR parsing 3
5. 5. LR Parsing LR Parsing I 4
6. 6. idea LR Parsing LR parsing problems with LL parsing • predicting right rule • left recursion LR parsing • see whole right-hand side of a rule • look ahead • shift or reduce 5
7. 7. example LR Parsing LR parsing 6 * 3 \$ input \$ stack 7* 37 + S → E \$ E → E * E E → E + E E → Num grammar
8. 8. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + S → E \$ E → E * E E → E + E E → Num
9. 9. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + S → E \$ E → E * E E → E + E E → Num
10. 10. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + * 3 \$7+\$ E 3* S → E \$ E → E * E E → E + E E → Num
11. 11. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + * 3 \$7+\$ E 3* S → E \$ E → E * E E → E + E E → Num
12. 12. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + * 3 \$7+\$ E 3* S → E \$ E → E * E E → E + E E → Num
13. 13. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + * 3 \$7+\$ E 3* * 3 \$7+\$ E * E S → E \$ E → E * E E → E + E E → Num
14. 14. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + * 3 \$7+\$ E 3* * 3 \$7+\$ E * E * 3 \$7+\$ E S → E \$ E → E * E E → E + E E → Num
15. 15. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + * 3 \$7+\$ E 3* * 3 \$7+\$ E * E * 3 \$7+\$ E S → E \$ E → E * E E → E + E E → Num
16. 16. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + * 3 \$7+\$ E 3* * 3 \$7+\$ E * E * 3 \$7+\$ E S → E \$ E → E * E E → E + E E → Num
17. 17. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + * 3 \$7+\$ E 3* * 3 \$7+\$ E * E * 3 \$7+\$ E \$ E + E 3* \$ S → E \$ E → E * E E → E + E E → Num
18. 18. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + * 3 \$7+\$ E 3* * 3 \$7+\$ E * E * 3 \$7+\$ E \$ E + E 3* \$ S → E \$ E → E * E E → E + E E → Num
19. 19. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + * 3 \$7+\$ E 3* * 3 \$7+\$ E * E * 3 \$7+\$ E \$ E + E 3* \$ S → E \$ E → E * E E → E + E E → Num
20. 20. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + * 3 \$7+\$ E 3* * 3 \$7+\$ E * E * 3 \$7+\$ E \$ E + E 3* \$ \$ E + E * E \$ S → E \$ E → E * E E → E + E E → Num
21. 21. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + * 3 \$7+\$ E 3* * 3 \$7+\$ E * E * 3 \$7+\$ E \$ E + E 3* \$ \$ E + E \$ E + E * E \$ \$ S → E \$ E → E * E E → E + E E → Num
22. 22. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + * 3 \$7+\$ E 3* * 3 \$7+\$ E * E * 3 \$7+\$ E \$ E + E 3* \$ \$ E + E \$ E + E * E \$ \$ \$ E \$ S → E \$ E → E * E E → E + E E → Num
23. 23. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + * 3 \$7+\$ E 3* * 3 \$7+\$ E * E * 3 \$7+\$ E \$ E + E 3* \$ \$ E + E \$ E + E * E \$ \$ \$ E \$ S → E \$ E → E * E E → E + E E → Num
24. 24. example LR Parsing LR parsing 6 * 3 \$\$ 7* 37 + * 3 \$7+\$ E 3* * 3 \$7+\$ E * E * 3 \$7+\$ E \$ E + E 3* \$ \$ E + E \$ E + E * E \$ \$ \$ E \$ \$ S S → E \$ E → E * E E → E + E E → Num
25. 25. LR Parsing Grammar classes 7 context-free grammars LL(k) LL(1) LL(0)
26. 26. LR Parsing Grammar classes 7 context-free grammars LR(0) LL(k) LL(1) LL(0)
27. 27. LR Parsing Grammar classes 7 context-free grammars LR(1) LR(0) LL(k) LL(1) LL(0)
28. 28. LR Parsing Grammar classes 7 context-free grammars LR(k) LR(1) LR(0) LL(k) LL(1) LL(0)
29. 29. LR Parsing Grammar classes 7 context-free grammars LR(k) LR(1) SLR LR(0) LL(k) LL(1) LL(0)
30. 30. LR Parsing Grammar classes 7 context-free grammars LR(k) LR(1) LALR(1) SLR LR(0) LL(k) LL(1) LL(0)
31. 31. LR Parsing LR Parse Tables II 8
32. 32. parse table LR Parsing 9 rows • states of a DFA columns • topmost stack symbol • Σ, N entries • reduce, rule number • shift, goto state • goto state • accept state LR parsing T1 ... N1 ... 1 s 3 2 g 5 3 r 1 4 r 2 a 5 6 g 1 7 s 1 8 ...
33. 33. items, closure & goto LR Parsing LR(0) parse tables 10 S → x S → ( L ) L → S L → L , S
34. 34. S’ → . S \$ items, closure & goto LR Parsing LR(0) parse tables 10 S → x S → ( L ) L → S L → L , S
35. 35. S’ → . S \$ items, closure & goto LR Parsing LR(0) parse tables 10 item S → x S → ( L ) L → S L → L , S
36. 36. closure • for every item A → α . X β • for every rule X → γ • add item X → . γ S’ → . S \$ items, closure & goto LR Parsing LR(0) parse tables 10 item S → x S → ( L ) L → S L → L , S
37. 37. S’ → . S \$ S → . x S → . ( L ) closure • for every item A → α . X β • for every rule X → γ • add item X → . γ items, closure & goto LR Parsing LR(0) parse tables 10 S → x S → ( L ) L → S L → L , S
38. 38. S’ → . S \$ S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S → x S → ( L ) L → S L → L , S
39. 39. S’ → . S \$ S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S’ → S . \$ S S → x S → ( L ) L → S L → L , S
40. 40. S’ → . S \$ S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S’ → S . \$ S x S → x . S → x S → ( L ) L → S L → L , S
41. 41. S’ → . S \$ S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S’ → S . \$ S x S → x . ( S → ( . L ) S → x S → ( L ) L → S L → L , S
42. 42. S’ → . S \$ S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S’ → S . \$ S x S → x . ( S → ( . L ) L → . S L → . L , S S → . x S → . ( L ) S → x S → ( L ) L → S L → L , S
43. 43. S’ → . S \$ S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S’ → S . \$ S x S → x . ( S → ( . L ) L → . S L → . L , S S → . x S → . ( L ) x S → x S → ( L ) L → S L → L , S
44. 44. S’ → . S \$ S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S’ → S . \$ S x S → x . ( S → ( . L ) L → . S L → . L , S S → . x S → . ( L ) x ( S → x S → ( L ) L → S L → L , S
45. 45. S’ → . S \$ S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S’ → S . \$ S x S → x . ( S → ( . L ) L → . S L → . L , S S → . x S → . ( L ) x ( S L → S . S → x S → ( L ) L → S L → L , S
46. 46. S’ → . S \$ S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S’ → S . \$ S x S → x . ( S → ( . L ) L → . S L → . L , S S → . x S → . ( L ) x ( S L → S . L S → ( L . ) L → L . , S S → x S → ( L ) L → S L → L , S
47. 47. S’ → . S \$ S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S’ → S . \$ S x S → x . ( S → ( . L ) L → . S L → . L , S S → . x S → . ( L ) x ( S L → S . L S → ( L . ) L → L . , S ) S → ( L ) . S → x S → ( L ) L → S L → L , S
48. 48. S’ → . S \$ S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S’ → S . \$ S x S → x . ( S → ( . L ) L → . S L → . L , S S → . x S → . ( L ) x ( S L → S . L S → ( L . ) L → L . , S ) S → ( L ) . , L → L , . S S → x S → ( L ) L → S L → L , S
49. 49. S’ → . S \$ S → . x S → . ( L ) L → L , . S S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S’ → S . \$ S x S → x . ( S → ( . L ) L → . S L → . L , S S → . x S → . ( L ) x ( S L → S . L S → ( L . ) L → L . , S ) S → ( L ) . , S → x S → ( L ) L → S L → L , S
50. 50. S’ → . S \$ S → . x S → . ( L ) L → L , . S S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S’ → S . \$ S x S → x . ( S → ( . L ) L → . S L → . L , S S → . x S → . ( L ) x ( S L → S . L S → ( L . ) L → L . , S ) S → ( L ) . , x S → x S → ( L ) L → S L → L , S
51. 51. S’ → . S \$ S → . x S → . ( L ) L → L , . S S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S’ → S . \$ S x S → x . ( S → ( . L ) L → . S L → . L , S S → . x S → . ( L ) x ( S L → S . L S → ( L . ) L → L . , S ) S → ( L ) . , x ( S → x S → ( L ) L → S L → L , S
52. 52. S’ → . S \$ S → . x S → . ( L ) L → L , . S S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S’ → S . \$ S x S → x . ( S → ( . L ) L → . S L → . L , S S → . x S → . ( L ) x ( S L → S . L S → ( L . ) L → L . , S ) S → ( L ) . , x ( S L → L , S . S → x S → ( L ) L → S L → L , S
53. 53. S’ → . S \$ S → . x S → . ( L ) L → L , . S S → . x S → . ( L ) items, closure & goto LR Parsing LR(0) parse tables 10 S’ → S . \$ S x S → x . ( S → ( . L ) L → . S L → . L , S S → . x S → . ( L ) x ( S L → S . L S → ( L . ) L → L . , S ) S → ( L ) . , x ( S L → L , S . S → x S → ( L ) L → S L → L , S 1 2 3 4 6 7 5 8 9
54. 54. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S
55. 55. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$)x,x( 1
56. 56. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$)x,x 1 3
57. 57. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$)x, 1 3 2
58. 58. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$)x, 1 3
59. 59. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$)x, 1 3 6
60. 60. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$)x, 1 3
61. 61. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$)x, 1 3 5
62. 62. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$)x 1 3 5 8
63. 63. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$) 1 3 5 8 2
64. 64. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$) 1 3 5 8
65. 65. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$) 1 3 5 8 9
66. 66. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$) 1 3
67. 67. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$) 1 3 5
68. 68. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$ 1 3 5 7
69. 69. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$ 1
70. 70. ( ) x , \$ S L 1 s 3 s 2 g 4 2 r 1 r 1 r 1 r 1 r 1 3 s 3 s 2 g 6 g 5 4 a 5 s 7 s 8 6 r 3 r 3 r 3 r 3 r 3 7 r 2 r 2 r 2 r 2 r 2 8 s 3 s 2 g 9 9 r 4 r 4 r 4 r 4 r 4 result LR Parsing 11 LR(0) parse tables S → x S → ( L ) L → S L → L , S \$ 1 4
71. 71. LR Parsing Conflict Resolution III 12
72. 72. E → T + . E E → . T + E E → . T T → . x S → . E \$ E → . T + E E → . T T → . x shift-reduce conflicts LR Parsing SLR parse tables 13 T → x . x E S → E . \$ T T + E E → T . + E E → T . E → T + E . E → T + E E → T T → x x
73. 73. E → T + . E E → . T + E E → . T T → . x S → . E \$ E → . T + E E → . T T → . x shift-reduce conflicts LR Parsing SLR parse tables 13 T → x . x E S → E . \$ T T + E E → T . + E E → T . E → T + E . E → T + E E → T T → x 1 2 3 5 4 6x
74. 74. E → T + . E E → . T + E E → . T T → . x S → . E \$ E → . T + E E → . T T → . x shift-reduce conflicts LR Parsing SLR parse tables 13 T → x . x E S → E . \$ T T + E E → T . + E E → T . E → T + E . E → T + E E → T T → x 1 2 3 5 4 6x x + \$ E T 1 s 5 g 2 g 3 2 a 3 r 2 ? r 2 4 s 5 g 6 g 3 5 r 3 r 3 r 3 6 r 1 r 1 r 1
75. 75. E → T + . E E → . T + E E → . T T → . x S → . E \$ E → . T + E E → . T T → . x shift-reduce conflicts LR Parsing SLR parse tables 13 T → x . x E S → E . \$ T T + E E → T . + E E → T . E → T + E . E → T + E E → T T → x 1 2 3 5 4 6x x + \$ E T 1 s 5 g 2 g 3 2 a 3 s 4 r 2 4 s 5 g 6 g 3 5 r 3 r 3 6 r 1 Reduce a production S → … on symbols k ∈ Σ, k ∈ Follow(S)
76. 76. look-ahead LR Parsing LR(1) parse tables 14 E → T + E E → T T → x S → . E \$ E → . T + E E → . T T → . x ? \$ \$ + \$ E S → E . \$ ? T E → T . + E E → T . \$ \$ T → x . x + \$ T + x E → T + . E E → . T + E E → . T T → . x \$ \$ \$ + \$ E E → T + E . \$ closure • for every item A → α . X β, z • for every rule X → γ • for every w ∈ First(βz) • add item X → . γ, w
77. 77. look-ahead LR Parsing LR(1) parse tables 14 E → T + E E → T T → x S → . E \$ E → . T + E E → . T T → . x ? \$ \$ + \$ E S → E . \$ ? T E → T . + E E → T . \$ \$ T → x . x + \$ T + x E → T + . E E → . T + E E → . T T → . x \$ \$ \$ + \$ E E → T + E . \$ x + \$ E T 1 s 5 g 2 g 3 2 a 3 s 4 r 2 4 s 5 g 6 g 3 5 r 3 r 3 6 r 1 closure • for every item A → α . X β, z • for every rule X → γ • for every w ∈ First(βz) • add item X → . γ, w
78. 78. state space reduction LR Parsing LALR(1) parse tables unify states • with same items • and same outgoing transitions • but different look-ahead sets might introduce new conflicts 15
79. 79. state space reduction LR Parsing LALR(1) parse tables 16 S’ → S \$ S → a E c S → a F d S → b F c S → b E d E → e F → e S' → . S \$ S → . a E c S → . a F d S → . b F c S → . b E d ? \$ \$ \$ \$
80. 80. state space reduction LR Parsing LALR(1) parse tables 17 S’ → S \$ S → a E c S → a F d S → b F c S → b E d E → e F → e S' → . S \$ S → . a E c S → . a F d S → . b F c S → . b E d ? \$ \$ \$ \$ S → a . E c S → a . F d E → . e F → . e \$ \$ c d a
81. 81. state space reduction LR Parsing LALR(1) parse tables 18 S’ → S \$ S → a E c S → a F d S → b F c S → b E d E → e F → e S' → . S \$ S → . a E c S → . a F d S → . b F c S → . b E d ? \$ \$ \$ \$ S → b . F c S → b . E d E → . e F → . e \$ \$ d c a b S → a . E c S → a . F d E → . e F → . e \$ \$ c d
82. 82. state space reduction LR Parsing LALR(1) parse tables 19 S’ → S \$ S → a E c S → a F d S → b F c S → b E d E → e F → e S' → . S \$ S → . a E c S → . a F d S → . b F c S → . b E d ? \$ \$ \$ \$ E → e . F → e . c d a b e S → b . F c S → b . E d E → . e F → . e \$ \$ d c S → a . E c S → a . F d E → . e F → . e \$ \$ c d
83. 83. state space reduction LR Parsing LALR(1) parse tables 20 S’ → S \$ S → a E c S → a F d S → b F c S → b E d E → e F → e S' → . S \$ S → . a E c S → . a F d S → . b F c S → . b E d ? \$ \$ \$ \$ E → e . F → e . c d a b e e E → e . F → e . d c S → b . F c S → b . E d E → . e F → . e \$ \$ d c S → a . E c S → a . F d E → . e F → . e \$ \$ c d
84. 84. state space reduction LR Parsing LALR(1) parse tables 21 S’ → S \$ S → a E c S → a F d S → b F c S → b E d E → e F → e S' → . S \$ S → . a E c S → . a F d S → . b F c S → . b E d ? \$ \$ \$ \$ E → e . F → e . c d a b e e E → e . F → e . d c S → b . F c S → b . E d E → . e F → . e \$ \$ d c S → a . E c S → a . F d E → . e F → . e \$ \$ c d
85. 85. state space reduction LR Parsing LALR(1) parse tables 22 S’ → S \$ S → a E c S → a F d S → b F c S → b E d E → e F → e S' → . S \$ S → . a E c S → . a F d S → . b F c S → . b E d ? \$ \$ \$ \$ S → a . E c S → a . F d E → . e F → . e \$ \$ c d S → b . F c S → b . E d E → . e F → . e \$ \$ d c E → e . F → e . {c, d} {c, d} a b e e
86. 86. state space reduction LR Parsing LALR(1) parse tables 23 S’ → S \$ S → a E c S → a F d S → b F c S → b E d E → e F → e S' → . S \$ S → . a E c S → . a F d S → . b F c S → . b E d ? \$ \$ \$ \$ E → e . F → e . {c, d} {c, d} a b e e Reduce/Reduce conflict! S → a . E c S → a . F d E → . e F → . e \$ \$ c d S → b . F c S → b . E d E → . e F → . e \$ \$ d c
87. 87. LR Parsing Generalized-LR Parsing IV 24
88. 88. Lexical Analysis Generalized Parsing • Parse all interpretations of the input, therefore it can handle ambiguous grammars. • Parsers split whenever finding an ambiguous interpretation and act in (pseudo) parallel. • Multiple parsers can join whenever they finish parsing an ambiguous fragment of the input. • Some parsers may "die", if the ambiguity was caused by a lack of lookahead. 25
89. 89. Lexical Analysis Generalized LR • Multiple parsers are synchronized on shift actions. • Each parser has its own stack, and as they share states, the overall structure becomes a graph (GSS). • If two parsers have the same state on top of their stack, they are joined into a single parser. • Reduce actions affect all possible paths from the top of the stack. 26
90. 90. Lexical Analysis Generalized LR 27 S → E \$ E → E + E E → E * E E → a SLR table
91. 91. Lexical Analysis Generalized LR 28 S → E \$ E → E + E E → E * E E → a S → . E \$ E → . E + E E → . E * E E → . a 0 SLR table
92. 92. Lexical Analysis Generalized LR 29 S → E \$ E → E + E E → E * E E → a S → . E \$ E → . E + E E → . E * E E → . a S → E . \$ E → E . + E E → E . * E E → a . E a 10 2 SLR table
93. 93. Lexical Analysis Generalized LR 30 S → E \$ E → E + E E → E * E E → a S → . E \$ E → . E + E E → . E * E E → . a S → E . \$ E → E . + E E → E . * E E → a . E → E + . E E → . E + E E → . E * E E → . a E → E * . E E → . E + E E → . E * E E → . a S → E \$ .E \$ a * + 10 3 2 4 5 SLR table
94. 94. Lexical Analysis Generalized LR 31 S → E \$ E → E + E E → E * E E → a S → . E \$ E → . E + E E → . E * E E → . a S → E . \$ E → E . + E E → E . * E E → a . E → E + . E E → . E + E E → . E * E E → . a E → E * . E E → . E + E E → . E * E E → . a E → E * E . E → E . + E E → E . * E S → E \$ .E \$ a * + E a 10 3 2 4 5 6 SLR table
95. 95. Lexical Analysis Generalized LR 32 S → E \$ E → E + E E → E * E E → a S → . E \$ E → . E + E E → . E * E E → . a S → E . \$ E → E . + E E → E . * E E → a . E → E + . E E → . E + E E → . E * E E → . a E → E * . E E → . E + E E → . E * E E → . a E → E + E . E → E . + E E → E . * E E → E * E . E → E . + E E → E . * E S → E \$ .E \$ a * + E E a a 10 3 2 4 5 7 6 SLR table
96. 96. Lexical Analysis Generalized LR 33 S → E \$ E → E + E E → E * E E → a S → . E \$ E → . E + E E → . E * E E → . a S → E . \$ E → E . + E E → E . * E E → a . E → E + . E E → . E + E E → . E * E E → . a E → E * . E E → . E + E E → . E * E E → . a E → E + E . E → E . + E E → E . * E E → E * E . E → E . + E E → E . * E S → E \$ .E \$ a * + E * E + a a 10 3 2 4 5 7 6 SLR table
97. 97. Lexical Analysis Generalized LR 34 S → E \$ E → E + E E → E * E E → a S → . E \$ E → . E + E E → . E * E E → . a S → E . \$ E → E . + E E → E . * E E → a . E → E + . E E → . E + E E → . E * E E → . a E → E * . E E → . E + E E → . E * E E → . a E → E + E . E → E . + E E → E . * E E → E * E . E → E . + E E → E . * E S → E \$ .E \$ a * + E * * + E + a a 10 3 2 4 5 7 6 SLR table
98. 98. Lexical Analysis Generalized LR 35 Nonter minal Nullable First Follow S E State Action Goto a + * \$ S E 0 1 2 3 4 5 6 7 (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a SLR table
99. 99. Lexical Analysis Generalized LR 36 Nonter minal Nullable First Follow S no a - E no a +, *, \$ State Action Goto a + * \$ S E 0 1 2 3 4 5 6 7 (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a SLR table
100. 100. Lexical Analysis Generalized LR 37 Nonter minal Nullable First Follow S no a - E no a +, *, \$ State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a SLR table
101. 101. Lexical Analysis Generalized LR 38 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 a + a * a 0 (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a
102. 102. Lexical Analysis Generalized LR 39 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 + a * a 0 2a (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a synchronize on shifts
103. 103. 0 2a Lexical Analysis Generalized LR 40 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 + a * a (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a
104. 104. Lexical Analysis Generalized LR 41 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 + a * a (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a 0 2a 1a : E
105. 105. 0 2a 1a : E 4 + Lexical Analysis Generalized LR 42 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 a * a (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a synchronize
106. 106. Lexical Analysis Generalized LR 43 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 a * a (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a 0 1a : E 4 +
107. 107. Lexical Analysis Generalized LR 44 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 * a (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a 0 1a : E 4 + 2 a synchronize
108. 108. 0 1a : E 4 + 2 a Lexical Analysis Generalized LR 45 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 * a (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a
109. 109. 0 1a : E 4 + 2 a a : E 7 Lexical Analysis Generalized LR 46 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 * a (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a
110. 110. Lexical Analysis Generalized LR 47 Parsing * a (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a 0 1a : E 4 + 2 a a : E 7 1a + a : E State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1
111. 111. 0 1a : E 4 + 2 a a : E 7 1a + a : E 5 ** Lexical Analysis Generalized LR 48 Parsing a (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a synchronize State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1
112. 112. Lexical Analysis Generalized LR 49 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a 0 1a : E 4 + a : E 7 1a + a : E 5 * * a
113. 113. 0 1a : E 4 + a : E 7 1a + a : E 5 * * 2 a Lexical Analysis Generalized LR 50 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a synchronize
114. 114. 0 1a : E 4 + a : E 7 1a + a : E 5 * * 2 a Lexical Analysis Generalized LR 51 Parsing (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1
115. 115. 0 1a : E 4 + a : E 7 1a + a : E 5 * * 6 a : E Lexical Analysis Generalized LR 52 Parsing (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1
116. 116. 0 1a : E 4 + a : E 7 1a + a : E 5 * * 6 a : E Lexical Analysis Generalized LR 53 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a
117. 117. 0 1a : E 4 + a : E 7 1a + a : E 5 * * 6 a : E 7 a * a : E Lexical Analysis Generalized LR 54 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a
118. 118. 0 1a : E 4 + a : E 7 1a + a : E 5 * * 6 a : E 7 a * a : E Lexical Analysis Generalized LR 55 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a
119. 119. 0 1a : E 4 + a : E 7 1a + a : E 5 * * 6 a : E 7 a * a : E 1 [a + a] * a : E Lexical Analysis Generalized LR 56 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a
120. 120. 0 1a : E 4 + a : E 7 1a + a : E 5 * * 6 a : E 7 a * a : E 1 [a + a] * a : E Lexical Analysis Generalized LR 57 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a
121. 121. [a + a] * a : E or a + [a * a] : E 0 1a : E 4 + a : E 7 1a + a : E 5 * * 6 a : E 7 a * a : E 1 Lexical Analysis Generalized LR 58 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a
122. 122. [a + a] * a : E or a + [a * a] : E 0 1a : E 4 + a : E 7 1a + a : E 5 * * 6 a : E 7 a * a : E 1 3 \$ Lexical Analysis Generalized LR 59 Parsing State Action Goto a + * \$ S E 0 s2 1 1 s4 s5 s3 2 r3 r3 r3 3 acc 4 s2 7 5 s2 6 6 s4/r2 s5/r2 r2 7 s4/r1 s5/r1 r1 (0) S → E \$ (1) E → E + E (2) E → E * E (3) E → a synchronize accept with trees on the link to initial state
123. 123. LR Parsing Scannerless Generalized-LR Parsing V 60
124. 124. Lexical Analysis Scannerless Generalized LR • Integrates scanning + parsing into a single GLR algorithm. • Normalization separates lexical and context-free symbols. • Crucial for avoiding conflicts when composing languages. • Introduces lexical ambiguities. Solution: ‣Follow Restrictions (implement longest match). ‣Reject Rules (implement reserved keywords). 61
125. 125. Lexical Analysis Scannerless Generalized LR 62 Normalization context-free syntax Exp.Add = <<Exp> + <Exp>> {left} Exp.Inc = <<Exp>++> Exp.ID = ID lexical syntax ID = [a-zA-Z] IDRest* IDRest = [a-zA-Z0-9] LAYOUT = [ tnr] ID = "let" {reject}
126. 126. context-free syntax Exp.Add = <<Exp> + <Exp>> {left} Exp.Inc = <<Exp>++> Exp.ID = ID lexical syntax ID = [a-zA-Z] IDRest* IDRest = [a-zA-Z0-9] LAYOUT = [ tnr] ID = "let" {reject} Lexical Analysis Scannerless Generalized LR 63 Normalization Exp-CF.Add = Exp-CF LAYOUT?-CF "+" LAYOUT?-CF Exp-CF {left} Exp-CF.Inc = Exp-CF LAYOUT?-CF "++" Exp-CF.ID = ID-CF ID-LEX = [a-zA-Z] IDRest*-LEX IDRest*-LEX = [a-zA-Z0-9] ID-LEX = "let" {reject} LAYOUT-CF = LAYOUT-LEX Separate lexical from context-free symbols and separate symbols in context-free syntax by optional layout.
127. 127. Lexical Analysis Scannerless Generalized LR 64 Normalization LAYOUT-CF = LAYOUT-LEX IDRest-CF = IDRest-LEX IDRest*-CF = IDRest*-LEX ID-CF = ID-LEX context-free syntax Exp.Add = <<Exp> + <Exp>> {left} Exp.Inc = <<Exp>++> Exp.ID = ID lexical syntax ID = [a-zA-Z] IDRest* IDRest = [a-zA-Z0-9] LAYOUT = [ tnr] ID = "let" {reject} Create injections from lexical to context-free symbols
128. 128. Lexical Analysis Scannerless Generalized LR 65 Normalization "+" = [43] "++" = [43] [43] "let" = [108] [101] [116] ID-LEX = [65-9097-122] IDRest*-LEX IDRest*-LEX = [48-5765-9097-122] LAYOUT-LEX = [9-101332] context-free syntax Exp.Add = <<Exp> + <Exp>> {left} Exp.Inc = <<Exp>++> Exp.ID = ID lexical syntax ID = [a-zA-Z] IDRest* IDRest = [a-zA-Z0-9] LAYOUT = [ tnr] ID = "let" {reject} Normalize literals symbols and character classes.
129. 129. Lexical Analysis Scannerless Generalized LR 66 Normalization LAYOUT?-CF = LAYOUT-CF LAYOUT?-CF = IDRest+-LEX = IDRest-LEX IDRest+-LEX = IDRest+-LEX IDRest-LEX IDRest*-LEX = IDRest*-LEX = IDRest+-LEX IDRest+-CF = IDRest+-LEX context-free syntax Exp.Add = <<Exp> + <Exp>> {left} Exp.Inc = <<Exp>++> Exp.ID = ID lexical syntax ID = [a-zA-Z] IDRest* IDRest = [a-zA-Z0-9] LAYOUT = [ tnr] ID = "let" {reject} Normalize regular expressions.
130. 130. Lexical Analysis Scannerless Generalized LR 67 Normalization context-free start-symbols Exp <START> = LAYOUT?-CF Exp-CF LAYOUT?-CF <Start> = <START> [256] Define extra rules for the start symbols.
131. 131. Lexical Analysis Scannerless Generalized LR 68 Parse Table Generation • Generation is based on SLR(1) item-sets. • Uses character classes instead of tokens. • Follow sets and goto actions are calculated for productions instead of non-terminals.
132. 132. Lexical Analysis Scannerless Generalized LR 69 Lexical Disambiguation • A reject rule is of the form: ID = "let" {reject} • When SGLR does a reduction with a reject rule, it marks the link as rejected. • Further action on a stack is forbidden whenever all links to it are rejected. • Follow restrictions are implemented as filters on the follow sets of productions.
133. 133. Lexical Analysis Generalized LR 70 • Priority rules: applied at parse table generation. E → E * . E E → . E + E E → . E * E E → . a multiplication has higher priority than addition! Context-free Disambiguation
134. 134. Lexical Analysis Generalized LR 71 • Priority rules: applied at parse table generation. multiplication is left associative! E → E * . E E → . E + E E → . E * E E → . a Context-free Disambiguation
135. 135. Lexical Analysis Generalized LR 72 • Priority rules: applied at parse table generation. • Disambiguation filters: applied after parsing (prefer and avoid). multiplication is left associative! E → E * . E E → . E + E E → . E * E E → . a Context-free Disambiguation
136. 136. LR Parsing Summary VI 73
137. 137. lessons learned LR Parsing Summary 74
138. 138. lessons learned LR Parsing Summary How can we generate LR parse tables? • items, closure, goto 74
139. 139. lessons learned LR Parsing Summary How can we generate LR parse tables? • items, closure, goto How can we improve LR(0) parse table generation? • SLR: consider FOLLOW sets to avoid shift-reduce conflicts • LR(1): consider look-ahead in states • LALR(1): unify LR(1) states to reduce state space 74
140. 140. lessons learned LR Parsing Summary How can we generate LR parse tables? • items, closure, goto How can we improve LR(0) parse table generation? • SLR: consider FOLLOW sets to avoid shift-reduce conflicts • LR(1): consider look-ahead in states • LALR(1): unify LR(1) states to reduce state space How can we handle conflicts in the parse table? • generalized parsing - supports all class of context free grammars • scannerless generalized LR - allow for proper language composition. 74