• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Compiler Components and their Generators - LR Parsing
 

Compiler Components and their Generators - LR Parsing

on

  • 1,757 views

Presentation slides for lecture 13 of course IN4303 on Compiler Construction at TU Delft.

Presentation slides for lecture 13 of course IN4303 on Compiler Construction at TU Delft.

Statistics

Views

Total Views
1,757
Views on SlideShare
1,375
Embed Views
382

Actions

Likes
1
Downloads
75
Comments
0

3 Embeds 382

https://blackboard.tudelft.nl 242
http://blackboard.tudelft.nl 126
http://www.programcreek.com 14

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • computer science: lexical syntax\n\ncan write this as a regular grammar\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • non-intuitive classification\nnon-intuitive error messages\ncomplex parsing algorithms\n
  • \n
  • \n
  • \n
  • declarative disambiguation\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • interleaved with evaluation of the parser\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • not closed under composition\n
  • not closed under composition\n
  • not closed under composition\n
  • \n
  • overlapping morphem classes\nglobally reserved words\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • what did before the scanner, now does the parser\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • closed under composition\n
  • closed under composition\n
  • closed under composition\n
  • closed under composition\n
  • closed under composition\n
  • closed under composition\n
  • closed under composition\n
  • closed under composition\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Compiler Components and their Generators - LR Parsing Compiler Components and their Generators - LR Parsing Presentation Transcript

  • Compiler Components & GeneratorsTraditional Parsing AlgorithmsGuido Wachsmuth Delft Course IN4303 University of Technology Compiler Construction Challenge the future
  • Recap: Traditional Parsing Algorithmslessons learnedHow can we parse context-free languages effectively? • predictive parsing algorithms • LR parsing algorithmsWhich grammar classes are supported by these algorithms? • LL(k) grammars, LL(k) languages • LR(k) grammars, LR(k) languagesHow can we generate compiler tools from that? • implement automata • generate parse tables Traditional Parsing Algorithms 2
  • Overviewtoday’s lecture Lexical Analysis 3
  • Overviewtoday’s lectureefficient parsing algorithms • LR parse table generation • SLR & LALR parse tables Lexical Analysis 3
  • Overviewtoday’s lectureefficient parsing algorithms • LR parse table generation • SLR & LALR parse tablesplagues of traditional parsing algorithms • paradise lost • paradise regained Lexical Analysis 3
  • ILR parse tables Lexical Analysis 4
  • Recap: LR parsingexample stack input$ 7 * 3 + 7 * 3 $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $$ E + E * E $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $$ E + E * E $$ E + E $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $$ E + E * E $$ E + E $$ E $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $$ E + E * E $$ E + E $$ E $ Traditional Parsing Algorithms 5
  • Recap: LR parsingexample$ 7 * 3 + 7 * 3 $$ E * 3 + 7 * 3 $$ E * E + 7 * 3 $$ E + 7 * 3 $$ E + E * 3 $$ E + E * E $$ E + E $$ E $$ S Traditional Parsing Algorithms 5
  • Recap: LR parsingparse tablerows T1 ... N1 ... • states of a DFA 1 s3 2 a g5columns 3 r1 • topmost stack symbol, Σ, N 4 r2entries 5 • reduce, rule number 6 g1 • shift, goto state 7 s1 • goto state 8 • accept ... Traditional Parsing Algorithms 6
  • LR(0) parse tablesitems, closure & goto S→x S→(L) L→S L→L,S Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & gotoS’ → . S $ S→x S→(L) L→S L→L,S Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto itemS’ → . S $ S→x S→(L) L→S L→L,S Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto item closure • for every item A → α . X βS’ → . S $ • for every rule X → γ • add item X → . γ S→x S→(L) L→S L→L,S Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto closure • for every item A → α . X βS’ → . S $S→.x • for every rule X → γS→.(L) • add item X → . γ S→x S→(L) L→S L→L,S Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & gotoS’ → . S $S→.xS→.(L) S→x S→(L) L→S L→L,S Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & gotoS’ → . S $S→.xS→.(L) S S→x S→(L) L→S L→L,SS’ → S . $ Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.xS→.(L) S S→x S→(L) L→S L→L,SS’ → S . $ Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.xS→.(L) ( S→(.L) S S→x S→(L) L→S L→L,SS’ → S . $ Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.xS→.(L) ( S→(.L) L→.S L→.L,S S→.x S→x S S→.(L) S→(L) L→S L→L,SS’ → S . $ Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.x xS→.(L) ( S→(.L) L→.S L→.L,S S→.x S→x S S→.(L) S→(L) L→S L→L,SS’ → S . $ Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.x xS→.(L) ( S→(.L) L→.S L→.L,S ( S→.x S S→x S→.(L) S→(L) L→S L→L,SS’ → S . $ Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.x xS→.(L) ( S→(.L) L→.S L→.L,S ( S→.x S S→x S→.(L) S→(L) S L→S L→L,SS’ → S . $ L→S. Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.x xS→.(L) ( S→(.L) L→.S L→.L,S ( S→.x S L S→(L.) S→x S→.(L) L→L.,S S→(L) S L→S L→L,SS’ → S . $ L→S. Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.x xS→.(L) ( S→(.L) L→.S L→.L,S ( S→.x S L S→(L.) S→x S→.(L) L→L.,S S→(L) S ) L→S L→L,SS’ → S . $ L→S. S→(L). Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $S→.x xS→.(L) ( L→L,.S S→(.L) L→.S , L→.L,S ( S→.x S L S→(L.) S→x S→.(L) L→L.,S S→(L) S ) L→S L→L,SS’ → S . $ L→S. S→(L). Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto x S→x.S’ → . S $ L→L,.SS→.x x S→.xS→.(L) ( S→.(L) S→(.L) L→.S , L→.L,S ( S→.x S L S→(L.) S→x S→.(L) L→L.,S S→(L) S ) L→S L→L,SS’ → S . $ L→S. S→(L). Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto x x S→x.S’ → . S $ L→L,.SS→.x x S→.xS→.(L) ( S→.(L) S→(.L) L→.S , L→.L,S ( S→.x S L S→(L.) S→x S→.(L) L→L.,S S→(L) S ) L→S L→L,SS’ → S . $ L→S. S→(L). Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto x x S→x.S’ → . S $ L→L,.SS→.x x S→.xS→.(L) ( ( S→.(L) S→(.L) L→.S , L→.L,S ( S→.x S L S→(L.) S→x S→.(L) L→L.,S S→(L) S ) L→S L→L,SS’ → S . $ L→S. S→(L). Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto L→L,S. S x x S→x.S’ → . S $ L→L,.SS→.x x S→.xS→.(L) ( ( S→.(L) S→(.L) L→.S , L→.L,S ( S→.x S L S→(L.) S→x S→.(L) L→L.,S S→(L) S ) L→S L→L,SS’ → S . $ L→S. S→(L). Traditional Parsing Algorithms 7
  • LR(0) parse tablesitems, closure & goto L→L,S.9 S x x S→x. 2S’ → . S $ 1 L→L,.S8S→.x x S→.xS→.(L) ( ( S→.(L) S→(.L)3 L→.S , L→.L,S ( S→.x S L S→(L.)5 S→x S→.(L) L→L.,S S→(L) S ) L→S L→L,SS’ → S . $ 4 L→S. 6 S→(L).7 Traditional Parsing Algorithms 7
  • LR(0) parse tablesresult ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 7 r2 r2 r2 r2 r2 L→L,S 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult ( x , x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult x , x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult , x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) 2 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult , x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult , x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) 6 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult , x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult , x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) 5 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult x ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 8 S→x 6 r3 r3 r3 r3 r3 S→(L) 5 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 2 5 s7 s8 8 S→x 6 r3 r3 r3 r3 r3 S→(L) 5 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 8 S→x 6 r3 r3 r3 r3 r3 S→(L) 5 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 9 5 s7 s8 8 S→x 6 r3 r3 r3 r3 r3 S→(L) 5 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult ) $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) 5 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 7 S→x 6 r3 r3 r3 r3 r3 S→(L) 5 L→S 3 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • LR(0) parse tablesresult $ ( ) x , $ S L 1 s3 s2 g4 2 r1 r1 r1 r1 r1 3 s3 s2 g6 g5 4 a 5 s7 s8 S→x 6 r3 r3 r3 r3 r3 S→(L) L→S 4 7 r2 r2 r2 r2 r2 L→L,S 1 8 s3 s2 g9 9 r4 r4 r4 r4 r4 Traditional Parsing Algorithms 8
  • SLR parse tablesshift-reduce conflicts E S→E.$S→.E$E→.T+EE→.T T E→T.+ET→.x E→T. T + x E→T+.E E→.T+E E→T+E E→.T E→TT→x. x T→.x E E→T+E. T→x Traditional Parsing Algorithms 9
  • SLR parse tablesshift-reduce conflicts E S→E.$ 2S→.E$E→.T+EE→.T T E→T.+ET→.x 1 E→T. 3 T + x E→T+.E E→.T+E E→T+E E→.T E→TT→x. x T→.x E E→T+E. 6 5 4 T→x Traditional Parsing Algorithms 9
  • SLR parse tablesshift-reduce conflicts x + $ E T 1 s5 g2 g3 E S→E.$ 2 2 aS→.E$E→.T+E 3 r2 ? r2E→.T T 4 s5 g6 g3 E→T.+ET→.x 1 5 r3 r3 r3 E→T. 3 6 r1 r1 r1 T + x E→T+.E E→.T+E E→T+E E→.T E→TT→x. x T→.x E E→T+E. 6 5 4 T→x Traditional Parsing Algorithms 9
  • SLR parse tablesshift-reduce conflicts x + $ E T 1 s5 g2 g3 E S→E.$ 2 2 aS→.E$E→.T+E 3 s4 r2E→.T T 4 s5 g6 g3 E→T.+ET→.x 1 5 r3 r3 E→T. 3 6 r1 T + x E→T+.E E→.T+E E→T+E E→.T E→TT→x. x T→.x E E→T+E. 6 5 4 T→x Traditional Parsing Algorithms 9
  • LR(1) parse tableslook-ahead E S→E.$ ?S→.E$ ?E→.T+E $E→.T $ T E→T.+E $T→.x +$ E→T. $ T + x E→T+.E $ E→.T+E $ E→T+E E→.T $ E→TT→x. +$ x T→.x +$ E E→T+E. $ T→x Traditional Parsing Algorithms 10
  • LR(1) parse tableslook-ahead x + $ E T 1 s5 g2 g3 E S→E.$ ? 2 aS→.E$ ?E→.T+E $ 3 s4 r2E→.T $ T 4 s5 g6 g3 E→T.+E $T→.x +$ 5 r3 r3 E→T. $ 6 r1 T + x E→T+.E $ E→.T+E $ E→T+E E→.T $ E→TT→x. +$ x T→.x +$ E E→T+E. $ T→x Traditional Parsing Algorithms 10
  • LALR(1) parse tablesstate space reductionunify states • with same items • and same outgoing transitions • but different look-ahead setsmight introduce new conflicts Traditional Parsing Algorithms 11
  • coffee break Traditional Parsing Algorithms 12
  • IIparadise lost & regained Lexical Analysis 13
  • PARADISE LOST
  • EFFICIENCY
  • THE FIRST PLAGUE WEREGRAMMAR CLASSES
  • context-free grammars
  • context-free grammars LL(0)
  • context-free grammars LL(1) LL(0)
  • context-free grammars LL(k) LL(1) LL(0)
  • context-free grammars LL(k) LL(1) LL(0) LR(0)
  • context-free grammars LL(k) LL(1) LR(1) LL(0) LR(0)
  • context-free grammars LL(k) LR(k) LL(1) LR(1) LL(0) LR(0)
  • context-free grammars LL(k) LR(k) LL(1) LR(1) SLR LL(0) LR(0)
  • context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
  • Exp “+” Exp ➝ Exp Term (“+” Term)* ➝ ExpExp “*” Exp ➝ Exp Fact (“*” Fact)* ➝ Term Num ➝ Exp Num ➝ Fact paradise grammar classes
  • Exp * Exp * Exp Term Term TermExp Exp Exp Fact Fact Fact3 + 7 + 21 3 + 7 + 21 paradise grammar classes
  • THE SECOND PLAGUE WASDISAMBIGUATION
  • Exp Exp Exp ExpExp Exp Exp Exp Exp Exp3 * 7 + 21 3 * 7 + 21
  • precedence operators associativity 1 ( ), [ ] non-associative 2 new non-associative 3 . left-associative 4 ++, -- non-associative 5 -, +, !, ~, ++, --, (type) right-associative 6 *, /, % left-associative 7 +, - left-associative … … … text books
  • context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
  • context-free grammars unambigous LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
  • Exp “+” Term ➝ Exp Term ➝ Exp Term “*” Fact ➝ Term Fact ➝ Term Num ➝ Factgrammar classes
  • FALSE PROPHETS
  • L = {ab, a} L = {ab, a}“a” “b” ➝ A “a” “b” / “a” ➝ A “a” ➝Aparadise PEGs
  • L = {ab, a} L = {a} “a” ➝ A “a” /“a” “b” ➝ A “a” “b” ➝ Aparadise PEGs
  • if c1 then if c2 then s1 else s2 dangling else
  • “if ” E “then” S “else” S / “if ” E “then” S /“if ” E “then” S ➝S “if ” E “then” S “else” S ➝ S PEGs
  • THE THIRD PLAGUE WASLEXICAL SYNTAX
  • morphology & syntax
  • LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)limited look-ahead
  • scanners
  • 3 * 7 + 21
  • 3 * 7 + 21
  • parsers
  • 3 * 7 + 21
  • Exp ExpExp Exp Exp3 * 7 + 21
  • x = 1. * .10y : array [ 1 .. 10 ] of integer
  • x = 1. * .10y : array [ 1 .. 10 ] of integer
  • x = 1. * .10y : array [ 1. .10 ] of integer
  • THE FOURTH PLAGUE WASTREE CONSTRUCTION
  • Exp Add Exp AddExp Exp Exp Const Const Const3 + 7 + 21 3 7 21 paradise
  • Exp * Add * AddTerm Term Term Const Const ConstFact Fact Fact 3 7 21 3 + 7 + 21 grammar classes
  • expr: INTEGER { $$ = con($1); }| expr + expr { $$ = opr(+, 2, $1, $3);}| expr * expr { $$ = opr(*, 2, $1, $3);}; semantic actions
  • THE FIFTH PLAGUE WAS EVOLUTION
  • Exp “+” Term ➝ ExpExp “+” Exp ➝ Exp Term ➝ ExpExp “*” Exp ➝ Exp Term “*” Fact ➝ Term Num ➝ Exp Fact ➝ Term Num ➝ Fact paradise grammar classes
  • CExp “+” Term ➝ CExp Term ➝ CExpExp “+” Exp ➝ Exp Term “*” Fact ➝ TermExp “*” Exp ➝ Exp Fact ➝ Term Num ➝ Exp Num ➝ FactExp “=” Exp ➝ Exp Exp “=” CExp ➝ ExpExp “<” Exp ➝ Exp Exp “<” CExp ➝ ExpExp “>” Exp ➝ Exp Exp “>” CExp ➝ Exp CExp ➝ Exp paradise grammar classes
  • THE SIXTH PLAGUE WASCOMPOSITION
  • parsers
  • context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
  • context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
  • context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
  • context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
  • scanners
  • public boolean authenticate(String user, String pw) { SQL stm = <| SELECT id FROM Users WHERE name = ${user} AND password = ${pw} |>; return executeQuery(stm).size() != 0;}
  • THE SEVENTH PLAGUE WASRESTRICTION TO PARSERS
  • PRETTY PRINTERS
  • SENTENCEGENERATORS
  • AST ACCESS
  • IDE SUPPORT
  • PARADISEREGAINED
  • GENERALISEDPARSING
  • context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
  • context-free grammars
  • context-free syntaxExp “+” Exp ➝ ExpExp “*” Exp ➝ Exp Exp "+" Exp -> Exp Num ➝ Exp Exp "*" Exp -> Exp NUM -> Exp paradise SDF
  • DECLARATIVEDISAMBIGUATION
  • context-free grammars unambigous LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
  • context-free grammars unambigous
  • context-free grammars
  • precedence operators associativity 1 ( ), [ ] non-associative 2 new non-associative 3 . left-associative 4 ++, -- non-associative 5 -, +, !, ~, ++, --, (type) right-associative 6 *, /, % left-associative 7 +, - left-associative … … … text books
  • context-free priorities Exp "*" Exp -> Exp {left}> Exp "+" Exp -> Exp {left}
  • context-free syntax "if" E "then" S -> S {prefer} "if" E "then" S "else" S -> S
  • SCANNERLESSPARSING
  • morphology & syntax
  • lexical syntax [0-9]+ -> NUM [ tn] -> LAYOUT "//" ~[n]* [n] -> LAYOUT
  • parser
  • parser
  • DECLARATIVETREE CONSTRUCTION
  • Exp Add Exp AddExp Exp Exp Const Const Const3 + 7 + 21 3 7 21 paradise
  • context-free syntax Exp "+" Exp -> Exp {cons("Add")} Exp "*" Exp -> Exp {cons("Mul")} NUM -> Exp {cons("Const")} SDF
  • Add Add( Add( Add Const("3"), Const("7")Const Const Const ), Const("21") 3 7 21 ) paradise SDF
  • SEAMLESSEVOLUTION
  • context-free syntax Exp "+" Exp -> Exp {cons("Add")} Exp "*" Exp -> Exp {cons("Mul")} NUM -> Exp {cons("Const")} Exp "=" Exp -> Exp {cons("Eq")} Exp ">" Exp -> Exp {cons("Gt")} Exp "<" Exp -> Exp {cons("Lt")}
  • MODULARCOMPOSITION
  • context-free grammars LL(k) LR(k) LL(1) LR(1) LALR(1) SLR LL(0) LR(0)
  • context-free grammars
  • public boolean authenticate(String user, String pw) { SQL stm = <| SELECT id FROM Users WHERE name = ${user} AND password = ${pw} |>; return executeQuery(stm).size() != 0;}
  • module Java-SQL imports Java SQL exports context-free syntax "<|" Query "|>" -> Exp {cons("ToSQL")} "${" Exp "}" -> SqlExp {cons("FromSQL")}
  • BEYONDPARSERS
  • PRETTY PRINTERS
  • SENTENCEGENERATORS
  • AST ACCESS
  • IDE SUPPORT
  • IIIsummary Lexical Analysis 88
  • Summarylessons learned Traditional Parsing Algorithms 89
  • Summarylessons learnedHow can we generate LR parse tables? • items, closure, goto Traditional Parsing Algorithms 89
  • Summarylessons learnedHow can we generate LR parse tables? • items, closure, gotoHow can we improve LR(0) parse table generation? • SLR: consider FOLLOW sets to avoid shift-reduce conflicts • LR(1): consider look-ahead in states • LALR(1): unify LR(1) states to reduce state space Traditional Parsing Algorithms 89
  • Summarylessons learnedHow can we generate LR parse tables? • items, closure, gotoHow can we improve LR(0) parse table generation? • SLR: consider FOLLOW sets to avoid shift-reduce conflicts • LR(1): consider look-ahead in states • LALR(1): unify LR(1) states to reduce state spaceWhy are efficient parsing algorithms problematic? • not longer pure, declarative, beautiful • paradise lost: seven plagues • paradise regained: scannerless generalised parsing Traditional Parsing Algorithms 89
  • Summarylessons learnedHow can we generate LR parse tables? • items, closure, gotoHow can we improve LR(0) parse table generation? • SLR: consider FOLLOW sets to avoid shift-reduce conflicts • LR(1): consider look-ahead in states • LALR(1): unify LR(1) states to reduce state spaceWhy are efficient parsing algorithms problematic? • not longer pure, declarative, beautiful • paradise lost: seven plagues • paradise regained: scannerless generalised parsing Traditional Parsing Algorithms 89
  • Literaturelearn more Traditional Parsing Algorithms 90
  • Literaturelearn moresyntactical analysis Andrew W. Appel, Jens Palsberg: Modern Compiler Implementation in Java, 2nd edition. 2002 Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman, Monica S. Lam: Compilers: Principles, Techniques, and Tools, 2nd edition. 2006 Traditional Parsing Algorithms 90
  • Literaturelearn moresyntactical analysis Andrew W. Appel, Jens Palsberg: Modern Compiler Implementation in Java, 2nd edition. 2002 Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman, Monica S. Lam: Compilers: Principles, Techniques, and Tools, 2nd edition. 2006generalised parsing Eelco Visser: Syntax Definition for Language Prototyping. PhD thesis 1997 M.G.J. van den Brand, J. Scheerder, J.J. Vinju, and E. Visser: Disambiguation Filters for Scannerless Generalized LR Parsers. CC 2002 Lennart C. L. Kats, Eelco Visser, Guido Wachsmuth: Pure and Declarative Syntax Definition - Paradise Lost and Regained. SPLASH 2010 Traditional Parsing Algorithms 90
  • Outlookcoming nextlectures • Lecture 14: Beyond grammarwareQuestion & Answer Jan 5 • 10 questions, submit & voteLab Dec 10 • support expressions • support statements • support ordinary classes & fields Traditional Parsing Algorithms 91
  • questions Lexical Analysis 92
  • credits Lexical Analysis 93
  • PicturescopyrightsSlide 1: Book Scanner by Ben Woosley, some rights reservedSlide 12: West Cornwall Pasty Co. by Dominica Williamson, some rights reservedSlide 91: Ostsee by Mario Thiel, some rights reservedSlide 92: Questions by Oberazzi, some rights reservedSlide 93: Too Much Credit by Andres Rueda, some rights reservedSlides 14-87: Pure and Declarative Syntax Definition: Paradise Lost and Regained, some rights reserved Traditional Parsing Algorithms 94