The document discusses the process of lexical analysis in compiler design. It explains that a lexical analyzer takes source code as input and outputs tokens. It uses two pointers - a begin pointer that points to the start of each token, and a forward pointer that moves through the input string character by character. The document then describes two approaches for buffering input - a single buffer scheme that has issues if a token spans the buffer, and a two buffer scheme that avoids this issue by using two buffers and switching between them.

1. Presentation by
Rushikesh kadam
SYSTEM
SOFTWARE

2. Lexical analyzer
Input string from
source code
Tokens,
lexims
int main( )
{
}
i n t m a i n ( ) { }

3. i n t m a i n ( ) { }
• Lexim begin pointer
• points to the beginning of every token
• Forward pointer
• Points every next character in the input string
Begin pointer
Forward
pointer
intially

4. i n t m a i n ( ) { }
Begin pointer
Forward
pointer
i n t m a i n ( ) { }
Begin pointer
Forward
pointer

5. i n t m a i n ( ) { }
Begin pointer
Forward
pointer
End of lexime

6. i n t m a i n ( ) { }
Begin pointer
Forward
pointer

7. i n t m a i n ( ) { }
Reading each
character
One system call
1000 character s 1000 system call

8. Buffering
• One Buffer Scheme
int i,j ;
A block of data is first read into a buffer, and then
second by lexical analyzer.
i n t i , j ;
In this scheme, only one buffer is used to store the input
string but the problem with this scheme is that if lexeme is
very long then it crosses the buffer boundary, to scan rest of
the lexeme the buffer has to be refilled, that makes
overwriting the first of lexeme.

9. • Two Buffer Scheme
int i,j ;
i=i+1;
i n t i , j ;
BP
FP
eof
Second buffer
i n t i , j ;
BP
FP
eof
First buffer
eof = end of file buffer
sentinal

10. • Algorithm
switch(*forward++)
{
}
case ‘ eof’:
if (forward is at the end of 1st buffer)
{
forward= beginning of 2nd buffer;
refill 2nd buffer;
}
else if (forward is at the end of 2nd buffer)
{
forward= beginning of 1st buffer;
refill 1st buffer;
}
else break;
case for other charcters :

11. THANK YOU!