Compiler Engineering Lab#5 : Symbol Table, Flex Tool

L A B # 5 : S Y M B O L TA B L E , F L E X
COMPILER
ENGINEERING

WHY DO I NEED A SYMBOL TABLE?
• Symbol Table answers the following questions:
1. For a certain declaration of an Identifier name , does it have
multiple declarations in different scopes?
• If YES, keep track of these different cases
2. For a USE of an Identifier name, to which scope does it
correspond? (using the "most closely nested" rule)
3. How can various language logical structures be presented?
• One of the Main Purposes of Symbol Table is: to keep
track of IDENTIFIERS recognized in the input stream.
• All subsequent references to identifiers refer to the
appropriate symbol table index.
1-4/4/12
Department of Computer Science -
Compiler Engineering Lab
2

WHY DO I NEED A SYMBOL TABLE?
• So, Symbol Table is a group of linked hash tables that
manages either:
• IDENTIFIERS attributes,
• or convey the structure of a statementexpression that they
reflect.
1-4/4/12
3

BUILDING COMPILER WITH
1-4/4/12
4
Lexical Analysis
(Lex Flex)
Syntax-Semantic
Analysis
(Yacc Bison)
Assembly
(LLVM)
Linking

1-4/4/12
5
TOKENS ARE NUMERICAL
REPRESENTATIONS OF STRINGS,
AND SIMPLIFY PROCESSING
First: Lexical Analysis with Flex
Reminder:
What is the goal of Lexical Analysis phase?
In other words, why do I use TOKENS?

BUILDING A COMPILER WITH
FLEX(LEX) & BISON(YACC)
1-4/4/12
6

FLEX & BISON INPUT-OUTPUT
INTERACTION
• Bas.y (Parser Input):
• Y.tab.h (Parser Output):
• is a header file that contains, among other things,
• the definitions of the token names NUM, OPA, etc.,
• and the variable yylval that we use to pass the bison code the semantic values of
tokens
• Y.tab.c (Parser Output):
• contains the CC++ code for the parser (which is a function called
yyparse())
• Bas.l (Lexica Analyzer Input):
• needs those definitions in (Y.tab.h) so that the yylex() function it defines for
bison can pass back the information it needs to pass back.
• Lex.yy.c(Lexica Analyzer Output): generated by FLEX that
contains, among other things, the definition of the yylex() function
1-4/4/12
7

FLEX
• Flex is an open-source tool for Fast lexical Analyzer can
be downloaded from Flex Download Webpage
• Flex reads user-specified input files
• If no input file is given, Flex will read its standard file
• Flex INPUT file:
• is a description of a scanner to generate
• The description is Pairs of Regular Expressions and C Code
called RULES
Source: Flex Website
1-4/4/12
8

FLEX
• When the executable runs it analyzes its input for
occurrences of text matching the Regular Expressions
(RegExp) for each rule.
• Whenever it finds a match  it executes the corresponding C
code
• Flex generates a C source file named “lex.yy.c” which
defines the function yylex()
• The file “lex.yy.c” can be compiled and linked to produce
an executable
1-4/4/12
9

BISON(1)
• Bison is a general-purpose parser generator that
converts an annotated context-free grammar into a
deterministic [LR or generalized LR (GLR)] parser.
• Bison, you can use it to develop a wide range of
language parsers.
• Bison is compatible with YACC
• Based on CC++ and works with Java.
(1) Source: Bison Webpage
1-4/4/12
10

1-4/4/12 11
FLEX TOOL
FIRST : LEXICAL ANALYSIS

BISONYACC & PARSING
• The grammar in the previous diagram (FlexBison
Interaction Model Diagram) is a text file you create with a
text editor
• Yacc will read your grammar and generate C code for a
syntax analyzer or parser
• The syntax analyzer uses grammar rules that allow it to
analyze tokens from the lexical analyzer and create a
syntax tree
• The syntax tree imposes a hierarchical structure on the
tokens
• e.g. operator precedence and associativity are apparent in the
syntax tree
1-4/4/12
12

LEXICAL ANALYSIS: FLEX IN DETAIL
• The flex generated scanner code tries to match
characters from the current input stream to these regular
expressions, and when a match is found, it executes the
associtated action code.
• The variable yytext contains the string (in the C sense,
i.e. '0' terminated char*.) of characters that were
matched.
• When more than one match is possible it breaks ties by
going with the longest match then first listed
1-4/4/12
13

FLEX FILES FORMAT
The general format of Lex source is:
{definitions}
%%
{rules}
%%
{user subroutines}
1-4/4/12
14

FLEX FILES FORMAT
• Any line which is not part of a Lex rule or action which begins
with a blank or tab is copied into the Lex generated program.
(used for Comments)
• source input prior to the first %% delimiter will be external to
any function in the code; if it appears immediately after the
first %%, it appears in an appropriate place for declarations in
the function written by
• Anything included between lines containing only %{ and %} is
copied out as above. The delimiters are discarded. This
format permits entering text like preprocessor statements that
must begin in column 1.
1-4/4/12
15

FLEX FILES FORMAT
• Any line which is not part of a Lex rule or action which begins
with a blank or tab is copied into the Lex generated program.
(used for Comments)
• This means each code-line must start from column one.
• source input prior to the first %% delimiter will be external to
any function in the code; if it appears immediately after the
first %%, it appears in an appropriate place for declarations in
the function written by
• Anything included between lines containing only %{ and %}
is copied out as above. The delimiters are discarded. This
format permits entering text like preprocessor statements that
must begin in column 1.
1-4/4/12
16

1-4/4/12
17

EXAMPLE # 1 : CALCULATOR FLEX FILE
Write a Flex file for a calculator that is able
to recognize the following:
• numbers,
• (+,-,*,/) operators,
• and parantheses
1-4/4/12
18

EXAMPLE # 1 : CALCULATOR (.L)
%{
#include "ex1.tab.hpp"
#include <iostream>
using namespace std;
%}
%option noyywrap
%%
[0-9]+ { yylval.val = atoi(yytext); return NUM; }
[+|-] { yylval.sym = yytext[0]; return OPA; }
[*|/] { yylval.sym = yytext[0]; return OPM; }
"(" { return LP; }
")" { return RP; }
";" { return STOP; }
<<EOF>> { return 0; }
[ tn]+ { } . { cerr << "Unrecognized token!" << endl; exit(1); }
%%
1-4/4/12
19

EXAMPLE # 2 : “BASIC LANGUAGE” FILE
Write a Flex file for ”Basic Language” that is
able to recognize the following:
• Numbers,
• Identifiers,
• Keywords,
• Relation Operations
• Arthimitic operators,
• and Delimiters
1-4/4/12
20

EXAMPLE # 2 : (BAS.L) FILE FOR BASIC
LANGAUGE
%{
#include "y.tab.h”
%}
digit [0-9]
letter [a-zA-Z]
%%
"+" { return PLUS; }
"-" { return MINUS; }
"*" { return TIMES; }
"/" { return SLASH; }
"(" { return LPAREN; }
")" { return RPAREN; }
";" { return SEMICOLON; }
"," { return COMMA; }
"." { return PERIOD; }
1-4/4/12
21

LANGAUGE
":=" { return BECOMES; }
"=" { return EQL; }
"<>" { return NEQ; }
"<" { return LSS; }
">" { return GTR; }
"<=" { return LEQ; }
">=" { return GEQ; }
1-4/4/12
22

LANGAUGE
"begin" { return BEGINSYM; }
"call" { return CALLSYM; }
"const" { return CONSTSYM; }
"do" { return DOSYM; }
"end" { return ENDSYM; }
"if" { return IFSYM; }
"odd" { return ODDSYM; }
"procedure" { return PROCSYM; }
"then" { return THENSYM; }
"var" { return VARSYM; }
"while" { return WHILESYM; }
1-4/4/12
23

LANGAUGE
{letter}({letter}|{digit})*
{ yylval.id = (char *)strdup(yytext);
return IDENT; }
{digit}+
{ yylval.num = atoi(yytext);
return NUMBER; }
[ tnr] /* skip whitespace */
. { printf("Unknown character [%c]n",yytext[0]);
return UNKNOWN; }
%% int yywrap(void) {return 1;}
1-4/4/12
24

INSTALLING FLEX & BISON
• Flex & Bison are Unix-based Tools that can be installed
directly under UnixLinux environment.
• Unser Windows Environment: Flex & Bison can be
installed under Cygwin ( Linux Terminal).
• Cygwin can be downloaded from here.
• Cygwin is a Linux API layer providing substantial Linux
API functionality.
1-4/4/12
25

INSTALLING FLEX & BISON
• Steps to download Cygwin:
1. Click the following link for Setup
2. Click Run  Next  Choose the option “Install from Internet”
 Next  Make sure the correct installation directory then
Next
3. Choose “Direct Connection” then Next
4. Choose any of the available HTTP download sites and click
Next
5. In the “Select Packages” page: search for both “Flex” and
“Bison” packages through the search textbox  then ensure
the “install” option is selected
6. Finish the installation process by clicking Next to the end.
1-4/4/12
26

OVERVIEW OF ANTLR(1)
• ANTLR, ANother Tool for Language Recognition, is a
language tool that provides a framework for constructing
recognizers, interpreters, compilers, and translators from
grammatical descriptions containing actions in a variety
of target languages.
• ANTLR provides excellent support for tree construction,
tree walking, translation, error recovery, and error
reporting
• ANTLR can be downloaded from the following link
ANTLR (.jar) file download link
(1) Source: ANTLR Webpage
1-4/4/12
27

QUESTIONS?
Thank you for listening 
1-4/4/12
28

Compiler Engineering Lab#5 : Symbol Table, Flex Tool

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Compiler Engineering Lab#5 : Symbol Table, Flex Tool

Similar to Compiler Engineering Lab#5 : Symbol Table, Flex Tool (20)

Recently uploaded

Recently uploaded (20)

Compiler Engineering Lab#5 : Symbol Table, Flex Tool