Lexical analysis is the first step in compiler construction and involves breaking a program into individual tokens. A Lex program has three sections: declarations, translation rules, and auxiliary code. The declarations section defines regular expressions and global variables. The translation rules section matches regular expressions to tokens and executes C code actions. When a regular expression is matched in the input stream, its corresponding action is executed to return the appropriate token.

1. Lexical Analysis

3. Lexical Analysis With Lex

4. Lexical analysis with Lex

5. Lex source program format
• The Lex program has three sections, separated
by %%:
declarations
%%
transition rules
%%
auxiliary code

6. Declarations section
• Code between %{ and }% is inserted directly into the lex.yy.c. Should
contain:
– Manifest constants (#define for each token)
– Global variables, function declarations, typedefs
• Outside %{ and }%, REGULAR DEFINITIONS are declared.
Examples:
delim [ tn]
ws {delim}+
letter [A-Za-z]
Each definition is a name
followed by a pattern.
Declared names can be used in
later patterns, if surrounded by
{ }.

7. Translation rules section
Translation rules take the form
p1 { action1 }
p2 { action2 }
… …
pn { actionn }
Where pi is a regular expression and actioni is a C program fragment to be
executed whenever pi is recognized in the input stream.
In regular expressions, references to regular definitions must be enclosed in
{} to distinguish them from the corresponding character sequences.