COMPILER CONSTRUCTION PRINCIPLES AND PRACTICE

COMPILER CONSTRUCTION Principles and Practice Kenneth C. Louden

2. Scanning (Lexical Analysis) PART ONE

Contents ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

The Function of a Scanner ,[object Object],[object Object],[object Object],[object Object],[object Object]

The Categories of Tokens ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Relationship between Tokens and its String ,[object Object],[object Object],[object Object]

Relationship between Tokens and its String ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Some Practical Issues of the Scanner ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Some Practical Issues of the Scanner ,[object Object],[object Object],[object Object],[object Object],RET 2 + 4 = ] x e d n i [ a 2 + 4 = ] x e d n i [ a

Some Relative Basic Concepts ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Some Relative Basic Concepts ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

More About Regular Expression ,[object Object],[object Object],[object Object]

2.2.1 Definition of Regular Expressions

Basic Regular Expressions ,[object Object],[object Object],[object Object],[object Object]

Regular Expression Operations ,[object Object],[object Object],[object Object]

Choice Among Alternatives ,[object Object],[object Object],[object Object],[object Object]

Concatenation ,[object Object],[object Object],[object Object],[object Object]

Repetition ,[object Object],[object Object],[object Object]

Precedence of Operation and Use of Parentheses ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Name for regular expression ,[object Object],[object Object],[object Object],[object Object]

Definition of Regular Expression ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Examples of Regular Expressions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Examples of Regular Expressions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Examples of Regular Expressions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Examples of Regular Expressions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],BACK

2.2.2 Extensions to Regular Expression

List of New Operations ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

List of New Operations ,[object Object],[object Object],[object Object],[object Object],[object Object],BACK

2.2.3 Regular Expressions for Programming Language Tokens

Number, Reserved word and Identifiers ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Comments ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Ambiguity ,[object Object],[object Object],[object Object]

White Space and Lookahead ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],RET

Introduction to Finite Automata ,[object Object],[object Object],[object Object],letter 1 letter digit 2

Introduction to Finite Automata ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],letter 1 letter digit 2

More About Finite Automata ,[object Object],[object Object],[object Object]

2.3.1 Definition of Deterministic Finite Automata

The Concept of DFA ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

The Concept of DFA ,[object Object],[object Object],[object Object],[object Object],[object Object]

Some differences between definition of DFA and the diagram: ,[object Object],[object Object],[object Object],[object Object],[object Object],letter start letter digit In-id

Examples of DFA ,[object Object],[object Object],b Not b Not b not b b not b

Examples of DFA ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],digit digit digit digit +  digit digit

Examples of DFA ,[object Object],[object Object],[object Object],[object Object],[object Object],+ digit digit digit . digit E + － digit digit E －

Examples of DFA ,[object Object],[object Object],BACK * 1 2 3 4 other other / * * / 5

2.3.2 Lookahead, Backtracking, and Nondeterministic Automata

A Typical Action of DFA Algorithm ,[object Object],[object Object],[object Object],start letter finish letter in_id digit [other] return ID

Finite automation for an identifier with delimiter and return value ,[object Object],[object Object],start letter finish letter in_id digit [other] return ID

Finite automation for an identifier with delimiter and return value ,[object Object],[object Object],start letter finish letter in_id digit [other] return ID letter start letter digit In-id

How to arrive at the start state in the first place (combine all the tokens into one DFA)

Each of these tokens begins with a different character ,[object Object],[object Object],[object Object],: < = = = return ASSIGN return LE return EQ : < = = = return ASSIGN return LE return EQ

Several tokens beginning with the same character ,[object Object],[object Object],< < = > < return NE return LT return LE < = > [other] return NE return LT return LE

Expand the Definition of a Finite Automaton ,[object Object],[object Object],[object Object],[object Object]

ε-transition ,[object Object],[object Object],[object Object],

ε-Transitions Used in Two Ways. ,[object Object],[object Object],[object Object],  : <  = = = 

Definition of NFA ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Some Notes ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Some Notes ,[object Object],[object Object]

Examples of NFAs ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],1 3 2 4 a b a   a b b b

Examples of NFAs ,[object Object],[object Object],[object Object],[object Object],BACK 1 2 3 5    4 6 a c 7   8  9  b 10 

2.3.3 Implementation of Finite Automata in Code

Ways to Translate a DFA or NFA into Code ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],1 letter letter 2 digit [other] 3

Ways to Translate a DFA or NFA into Code ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],* 1 2 3 4 other other / * * / 5

Ways to Translate a DFA or NFA into Code ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],1 letter letter 2 digit [other] 3

Ways to Translate a DFA or NFA into Code ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],* * 1 2 3 4 5 / * / other other

Ways to Translate a DFA or NFA into Code ,[object Object],[object Object],[object Object],Characters in the alphabet c States representing transitions T (s, c) States s

Ways to Translate a DFA or NFA into Code ,[object Object],[object Object],Brackets indicate “noninput-consuming” transitions This column indicates accepting states yes 3 no [3] 2 2 2 No 2 1 Accepting other digit letter Input char state

Ways to translate a DFA or NFA into Code ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],yes 5 no 3 4 5 4 no 3 4 3 3 no 3 2 no 2 1 Accepting Other * / Input char state

Features of Table-Driven Method ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],RET

COMPILER CONSTRUCTION PRINCIPLES AND PRACTICE

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to COMPILER CONSTRUCTION PRINCIPLES AND PRACTICE

Similar to COMPILER CONSTRUCTION PRINCIPLES AND PRACTICE (20)

More from bolovv

More from bolovv (7)

Recently uploaded

Recently uploaded (20)

COMPILER CONSTRUCTION PRINCIPLES AND PRACTICE