Language Processor Implementation
using Python
Objectiv
e


Implement parser, syntax analyzer, semantic
analyzer for Pascal language



Show main approaches to impleme...
Choosing Programming
Language

I investigated strong and week points of
several programming languages:
 Java
 Perl
 Pyt...
Python
Benefits

High level programming language
 Supports object oriented paradigm
 Convenient data types
 Relatively ...
Language Processor Work
Scheme
Entry

Parser

Syntax

Semantics

Code gen

Exit
Вихід
In memory tables
Parser.
Tokens
Identifier

^[A-Za-z][A-Za-z_0-9]{0,255}$
Integer Const
^[+-]?d{1,10}$
Float Const
^([+-]?((d+.d+)|(d+.d...
Algorythm of Analyzing
Lexems

def getClass(self, word):
c = None
if ((word in self.KeyWords)
or (word in metadata["delimi...
Attribute Table



Separate class



Design pattern Singleton



Uses hash table as internal structure



Fast access
...
Attribute Table Look
Syntax
Analyzer


Context free grammar



Left associated grammar



EBNF



Configuration format is yaml



Language...
YAML Configuration Format
Benefits


Short and readable



yaml uses data structures that are native to
programming lang...
Configuration Format of Language
Grammar

EBNF

rule
program ::=
EBNF

Program ID ; block .

rule in configuration
progr...
Semantic Analyzer. Semantic
Actions
Rule

without semantic actions
complex_action:
- [begin, action_list, end]
Rule

wit...
Attribute
classes
Making

code generation process easier, I created a
bunch of classes like AttrFor.

class

AttrFor (At...
Abstract Parse
Tree
Intermediary code
generation
We

are using tetrads language in order to generate
intermediary code
How tetrads language ...
Example of input file
program q;
var
a, b: integer;
i: integer;
d: integer;
begin
d := 4;
for i:= 1 to (2+2*2)*2 do
begin
...
Output of language processor using intermediary code

d:=4
i:=1
@Lid1:
if i > 12 goto @Lid2
b:=b + 1
a:=a * 2
i := i + 1
g...
Bottom line


I implemented parser, syntax analyzer, semantic analyzer,
intermediary code generation of Pascal programmin...
Upcoming SlideShare
Loading in …5
×

Language processor implementation using python

707 views
500 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
707
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Language processor implementation using python

  1. 1. Language Processor Implementation using Python
  2. 2. Objectiv e  Implement parser, syntax analyzer, semantic analyzer for Pascal language  Show main approaches to implementation of semantic analysis as well as intermediary code generation  Parameterize language processor
  3. 3. Choosing Programming Language I investigated strong and week points of several programming languages:  Java  Perl  Python  C++  Delphi As a result I chose Python due to several very good reasons.
  4. 4. Python Benefits High level programming language  Supports object oriented paradigm  Convenient data types  Relatively fast due to using C based libraries  Easy readable syntax  Cross-platform  Convenient tools for parsing YAML  Supports regular expressions out of the box 
  5. 5. Language Processor Work Scheme Entry Parser Syntax Semantics Code gen Exit Вихід In memory tables
  6. 6. Parser. Tokens Identifier ^[A-Za-z][A-Za-z_0-9]{0,255}$ Integer Const ^[+-]?d{1,10}$ Float Const ^([+-]?((d+.d+)|(d+.d+e[+-]d+)))$ String Literal ^'.{0,65535}'$
  7. 7. Algorythm of Analyzing Lexems def getClass(self, word): c = None if ((word in self.KeyWords) or (word in metadata["delimiters"]) or (word in metadata["double"]) or (word in metadata["conditional_delimiters"]) or (word in metadata["multiplicative"]) or (word in metadata["additive"]) or (word in metadata["Relation"])): c = word else: for r in self.RegExp.keys(): if re.compile(r).match(word): c = self.RegExp[r] return c
  8. 8. Attribute Table  Separate class  Design pattern Singleton  Uses hash table as internal structure  Fast access  Convenient format
  9. 9. Attribute Table Look
  10. 10. Syntax Analyzer  Context free grammar  Left associated grammar  EBNF  Configuration format is yaml  Language grammar is easily changed without source code modification
  11. 11. YAML Configuration Format Benefits  Short and readable  yaml uses data structures that are native to programming languages like Perl, Python
  12. 12. Configuration Format of Language Grammar EBNF rule program ::= EBNF Program ID ; block . rule in configuration programme: - [program, id, ;, Block, "."]
  13. 13. Semantic Analyzer. Semantic Actions Rule without semantic actions complex_action: - [begin, action_list, end] Rule with semantic actions complex_action: - [begin, "#200", action_list, end, "#220"]
  14. 14. Attribute classes Making code generation process easier, I created a bunch of classes like AttrFor. class AttrFor (AttrObject): self.parameter = None self.first = None self.last = None self.step = None self.body = None
  15. 15. Abstract Parse Tree
  16. 16. Intermediary code generation We are using tetrads language in order to generate intermediary code How tetrads language looks: Z := X op Y Z := op X Z := Y Z := Y[X] Z: GOTO Z If condition GOTO Z
  17. 17. Example of input file program q; var a, b: integer; i: integer; d: integer; begin d := 4; for i:= 1 to (2+2*2)*2 do begin b:=b + 1; a:=a * 2; end; d:=a; end.
  18. 18. Output of language processor using intermediary code d:=4 i:=1 @Lid1: if i > 12 goto @Lid2 b:=b + 1 a:=a * 2 i := i + 1 goto @Lid1 @Lid2: d:=a
  19. 19. Bottom line  I implemented parser, syntax analyzer, semantic analyzer, intermediary code generation of Pascal programming language  I showed main concepts of semantic analysis as well as intermediary code generation  Language processor has been parameterized

×