Download as PDF, PPTX
Uploaded byEelco Visser
Formal Grammars
The document discusses formal grammars and their applications in language specification and parsing. It introduces key concepts such as formal grammars, derivation, terminal and non-terminal symbols, and different types of grammars including context-sensitive, context-free and regular grammars. It also discusses applications of formal grammars in parsing and how they relate to theoretical computer science concepts like decidability and complexity of the word problem for different grammar types.
More Related Content
![RegularLanguageProperties [Autosaved].pptx](https://cdn.slidesharecdn.com/ss_thumbnails/regularlanguagepropertiesautosaved-230716173745-f8e4bc8d-thumbnail.jpg?width=640&height=640&fit=bounds)
Formal Grammars
- 1. IN4303 2016-2017 Compiler Construction FormalGrammars language specification Guido Wachsmuth, Eelco Visser
- 2. Formal Grammars 2 3* +7 21 3 * +7 21 Exp ExpExp Exp Exp parser
- 3. Formal Grammars 3 syntax definition errors parsegenerate check parse table generic parser
- 4. Formal Grammars Stratego DynSem NaBL2 SDF3 ESV editor SPT tests 4 syntax definition concretesyntax abstract syntax static semantics name binding type system dynamic semantics translation interpretation
- 5. Formal Grammars Stratego DynSem NaBL2 SDF3 ESV editor SPT tests 5 syntax definition concretesyntax abstract syntax static semantics name binding type system dynamic semantics translation interpretation
- 6. Formal Grammars Stratego DynSem NaBL2 SDF3 ESV editor SPT tests 6 syntax definition concretesyntax abstract syntax static semantics name binding type system dynamic semantics translation interpretation
- 7. Formal Grammars 7 PA R E N T A L ADVISORY THEORETICAL CONTENT
- 8. Formal Grammars 8 formallanguages
- 9. Formal Grammars 9 infiniteproductivity
- 10.
- 11.
- 12.
- 13.
- 14. Formal Grammars 13 vocabularyΣ finite, nonempty set of elements (words, letters) alphabet
- 15. Formal Grammars 13 vocabularyΣ finite, nonempty set of elements (words, letters) alphabet string over Σ finite sequence of elements chosen from Σ word, sentence, utterance
- 16. Formal Grammars 13 vocabularyΣ finite, nonempty set of elements (words, letters) alphabet string over Σ finite sequence of elements chosen from Σ word, sentence, utterance formal language λ set of strings over a vocabulary Σ λ ⊆ Σ*
- 17. Formal Grammars 14 formalgrammars
- 18. Formal Grammars 15 formalgrammar G derivation relation G formal language L(G) ⊆ Σ* L(G) = {w∈Σ* | S G * w}
- 19. Formal Grammars 16 G= (N, Σ, P, S) Num → Digit Num Num → Digit Digit → “0” Digit → “1” Digit → “2” Digit → “3” Digit → “4” Digit → “5” Digit → “6” Digit → “7” Digit → “8” Digit → “9” decimal numbers morphology
- 20. Formal Grammars 17 G= (N, Σ, P, S) Num → Digit Num Num → Digit Digit → “0” Digit → “1” Digit → “2” Digit → “3” Digit → “4” Digit → “5” Digit → “6” Digit → “7” Digit → “8” Digit → “9” decimal numbers morphology Σ: finite set of terminal symbols
- 21. Formal Grammars 18 G= (N, Σ, P, S) Num → Digit Num Num → Digit Digit → “0” Digit → “1” Digit → “2” Digit → “3” Digit → “4” Digit → “5” Digit → “6” Digit → “7” Digit → “8” Digit → “9” decimal numbers morphology Σ: finite set of terminal symbols N: finite set of non-terminal symbols
- 22. Formal Grammars 19 G= (N, Σ, P, S) Num → Digit Num Num → Digit Digit → “0” Digit → “1” Digit → “2” Digit → “3” Digit → “4” Digit → “5” Digit → “6” Digit → “7” Digit → “8” Digit → “9” decimal numbers morphology Σ: finite set of terminal symbols N: finite set of non-terminal symbols S∈N: start symbol
- 23. Formal Grammars 20 G= (N, Σ, P, S) Num → Digit Num Num → Digit Digit → “0” Digit → “1” Digit → “2” Digit → “3” Digit → “4” Digit → “5” Digit → “6” Digit → “7” Digit → “8” Digit → “9” decimal numbers morphology Σ: finite set of terminal symbols N: finite set of non-terminal symbols S∈N: start symbol P⊆N×(N∪Σ)* : set of production rules
- 24. Formal Grammars 21 Num DigitNum 4 Num 4 Digit Num 4 3 Num 4 3 Digit Num 4 3 0 Num 4 3 0 Digit 4 3 0 3 Num → Digit Num Digit → “4” Num → Digit Num Digit → “3” Num → Digit Num Digit → “0” Num → Digit Digit → “3” decimal numbers production
- 25. Formal Grammars 21 Num DigitNum 4 Num 4 Digit Num 4 3 Num 4 3 Digit Num 4 3 0 Num 4 3 0 Digit 4 3 0 3 Num → Digit Num Digit → “4” Num → Digit Num Digit → “3” Num → Digit Num Digit → “0” Num → Digit Digit → “3” decimal numbers production leftmost derivation
- 26. Formal Grammars 22 Num DigitNum Digit Digit Num Digit Digit Digit Num Digit Digit Digit Digit Digit Digit Digit 3 Digit Digit 0 3 Digit 3 0 3 4 3 0 3 Num → Digit Num Num → Digit Num Num → Digit Num Num → Digit Digit → “3” Digit → “0” Digit → “3” Digit → “4” decimal numbers production
- 27. Formal Grammars 22 Num DigitNum Digit Digit Num Digit Digit Digit Num Digit Digit Digit Digit Digit Digit Digit 3 Digit Digit 0 3 Digit 3 0 3 4 3 0 3 Num → Digit Num Num → Digit Num Num → Digit Num Num → Digit Digit → “3” Digit → “0” Digit → “3” Digit → “4” decimal numbers production rightmost derivation
- 28. Formal Grammars 23 G= (N, Σ, P, S) Exp → Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” binary expressions syntax
- 29. Formal Grammars 24 G= (N, Σ, P, S) Exp → Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” binary expressions syntax Σ: finite set of terminal symbols
- 30. Formal Grammars 25 G= (N, Σ, P, S) Exp → Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” binary expressions syntax Σ: finite set of terminal symbols N: finite set of non-terminal symbols
- 31. Formal Grammars 26 G= (N, Σ, P, S) Exp → Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” binary expressions syntax Σ: finite set of terminal symbols N: finite set of non-terminal symbols S∈N: start symbol
- 32. Formal Grammars 27 G= (N, Σ, P, S) Exp → Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” binary expressions syntax Σ: finite set of terminal symbols N: finite set of non-terminal symbols S∈N: start symbol P⊆N×(N∪Σ)* : set of production rules
- 33. Formal Grammars 28 Exp Exp+ Exp Exp + Exp * Exp 3 + Exp * Exp 3 + 4 * Exp 3 + 4 * 5 Exp → Exp “+” Exp Exp → Exp “*” Exp Exp → Num Exp → Num Exp → Num binary expressions production
- 34. Formal Grammars 29 formalgrammar G = (N, Σ, P, S) nonterminal symbols N terminal symbols Σ production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* start symbol S∈N
- 35. Formal Grammars 29 formalgrammar G = (N, Σ, P, S) nonterminal symbols N terminal symbols Σ production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* start symbol S∈N nonterminal symbol
- 36. Formal Grammars 29 formalgrammar G = (N, Σ, P, S) nonterminal symbols N terminal symbols Σ production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* start symbol S∈N context
- 37. Formal Grammars 29 formalgrammar G = (N, Σ, P, S) nonterminal symbols N terminal symbols Σ production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* start symbol S∈N replacement
- 38. Formal Grammars 29 formalgrammar G = (N, Σ, P, S) nonterminal symbols N terminal symbols Σ production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* start symbol S∈N
- 39. Formal Grammars 30 type-0,unrestricted: P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* type-1, context-sensitive: (a A c, a b c) type-2, context-free: P ⊆ N × (N∪Σ)* type-3, regular: (A, x) or (A, xB)
- 40. Formal Grammars 31 formalgrammars context-sensitive context-free regular
- 41. Formal Grammars 32 formalgrammar G derivation relation G formal language L(G) ⊆ Σ* L(G) = {w∈Σ* | S G * w}
- 42. Formal Grammars 33 formalgrammar G = (N, Σ, P, S) derivation relation G ⊆ (N∪Σ)* × (N∪Σ)* w G w’ ∃(p, q)∈P: ∃u,v∈(N∪Σ)* : w=u p v ∧ w’=u q v formal language L(G) ⊆ Σ* L(G) = {w∈Σ* | S G * w}
- 43. Formal Grammars 34 formallanguages context-sensitive context-free regular
- 44. Formal Grammars 35 3* +7 21 3 * +7 21 Exp ExpExp Exp Exp parser Derivation is about productivity. But what about parsing?
- 45. Formal Grammars 36 wordproblem
- 46. Formal Grammars 37 wordproblem χL: Σ* → {0,1} w → 1, if w∈L w → 0, else theoretical computer science decidability & complexity
- 47. Formal Grammars 37 wordproblem χL: Σ* → {0,1} w → 1, if w∈L w → 0, else decidability type-0: semi-decidable type-1, type-2, type-3: decidable theoretical computer science decidability & complexity
- 48. Formal Grammars 37 wordproblem χL: Σ* → {0,1} w → 1, if w∈L w → 0, else decidability type-0: semi-decidable type-1, type-2, type-3: decidable complexity type-1: PSPACE-complete type-2, type-3: P theoretical computer science decidability & complexity
- 49. Formal Grammars 37 wordproblem χL: Σ* → {0,1} w → 1, if w∈L w → 0, else decidability type-0: semi-decidable type-1, type-2, type-3: decidable complexity type-1: PSPACE-complete type-2, type-3: P theoretical computer science decidability & complexity PSPACE⊇NP⊇P
- 50. Formal Grammars 38 formalgrammars context-sensitive context-free regular theoretical computer science decidability & complexity
- 51. Formal Grammars 38 formalgrammars context-sensitive context-free regular theoretical computer science decidability & complexity
- 52. Formal Grammars 38 formalgrammars context-sensitive context-free regular theoretical computer science decidability & complexity
- 53. Formal Grammars 39 Exp→ Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” context-free grammars production vs. reduction rules
- 54. Formal Grammars 39 Exp→ Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” context-free grammars production vs. reduction rules productive form
- 55. Formal Grammars 39 Exp→ Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” Num → Exp Exp “+” Exp → Exp Exp “−” Exp → Exp Exp “*” Exp → Exp Exp “/” Exp → Exp “(” Exp “)” → Exp context-free grammars production vs. reduction rules productive form
- 56. Formal Grammars 39 Exp→ Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” Num → Exp Exp “+” Exp → Exp Exp “−” Exp → Exp Exp “*” Exp → Exp Exp “/” Exp → Exp “(” Exp “)” → Exp context-free grammars production vs. reduction rules productive form reductive form
- 57. Formal Grammars 40 3+ 4 * 5 Exp + 4 * 5 Exp + Exp * 5 Exp + Exp * Exp Exp + Exp Exp Num → Exp Num → Exp Num → Exp Exp “*” Exp → Exp Exp “+” Exp → Exp binary expressions reduction
- 58. Formal Grammars 41 3* +7 21 3 * +7 21 Exp ExpExp Exp Exp parser The word problem is about membership. But what about structure?
- 59.
- 60. Formal Grammars 43 Exp Num Exp +ExpExp Exp −Exp Exp Exp *Exp Exp Exp /Exp Exp Exp Exp( ) context-free grammars tree construction rules
- 61. Formal Grammars 44 binaryexpressions tree construction 3 + *4 5
- 62. Formal Grammars 44 binaryexpressions tree construction Exp 3 + *4 5
- 63. Formal Grammars 44 binaryexpressions tree construction Exp Exp 3 + *4 5
- 64. Formal Grammars 44 binaryexpressions tree construction Exp Exp 3 + *4 5 Exp
- 65. Formal Grammars 44 binaryexpressions tree construction Exp Exp Exp 3 + *4 5 Exp
- 66. Formal Grammars 44 binaryexpressions tree construction Exp Exp Exp 3 + *4 5 Exp Exp
- 67. Formal Grammars 45 binaryexpressions ambiguity Exp Exp Exp 3 + *4 5 Exp Exp Exp Exp Exp 3 + *4 5 Exp Exp
- 68. Formal Grammars 46 syntaxtrees different trees for same sentence derivations different leftmost derivations for same sentence different rightmost derivations for same sentence NOT just different derivations for same sentence context-free grammars ambiguity
- 69. Formal Grammars 47 parsetrees parent node: nonterminal symbol child nodes: terminal symbols abstract syntax trees (ASTs) abstract over terminal symbols convey information at parent nodes abstract over injective production rules syntax trees parse trees & abstract syntax trees
- 70. Formal Grammars 48 binaryexpressions parse tree & abstract syntax tree Exp Exp Exp Exp Exp (3 + *4 5 ) Exp
- 71. Formal Grammars 48 binaryexpressions parse tree & abstract syntax tree Const Mul Const 3 4 5 Const Add Exp Exp Exp Exp Exp (3 + *4 5 ) Exp
- 72. Formal Grammars 49 Exceptwhere otherwise noted, this work is licensed under
- 73. Formal Grammars 50 attribution slidetitle author license 1 The Pine, Saint Tropez Paul Signac public domain 2, 3, 35, 41 PICOL icons Melih Bilgil CC BY 3.0 9 Writing Caitlin Regan CC BY 2.0 10 Latin Grammar Anthony Nelzin 13, 15, 29-34, 38 Noam Chomsky Fellowsisters CC BY-NC-SA 2.0