• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Managed Compiler
 

Managed Compiler

on

  • 2,019 views

 

Statistics

Views

Total Views
2,019
Views on SlideShare
2,008
Embed Views
11

Actions

Likes
2
Downloads
26
Comments
0

2 Embeds 11

http://www.slideshare.net 6
http://www.linkedin.com 5

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Welcome and thank you all for coming. My name is Joao Magalhães, I’m a trainee from the Product Builder team and I’m here to give a presentation about the creation of compilers in a managed environment. For this presentation I’m using some of he work developed during my final (thesis) project for my Master degree.

Managed Compiler Managed Compiler Presentation Transcript

  • Language framework in a managed environment Microsoft Development Center Copenhagen Author: João Filipe Gama de Magalhães E-mail: t-joaode@microsoft.com February 2008
  • Overview
    • Objectives
    • Compiler Structure
    • Compiler Implementation
    • Developed Solution
    • Demo
    • Conclusions
    • Questions
    Language framework in a managed environment February 2008
  • Objectives
    • Introduce compiler construction
    • Differences between managed (.NET) compiler and native compiler
    • Introduce some of the used tools
    • Give directions on some of the most important design decisions
    Language framework in a managed environment February 2008
    • Constitutes the way to convert an input file (source) into a output (target) file in a process called translation
    • Composed by 4 fundamental parts (lexer, parser, semantic analysis and generation)
    Compiler Structure Language framework in a managed environment February 2008 Lexical Analyzer Semantic Analyzer Syntactical Analyzer Code Generator AST source file target file tokens
  • Compiler Implementation
    • Construction limited to managed (.NET) tools
    • Lexer and parser defined in LEX/YACC syntax (reuse old parsers and lexers)
    • C# backed parser, semantic analyzer and code generator
    • Utilization of patterns to simplify work in semantic analysis and code generation
    Language framework in a managed environment February 2008 Overview
  • Compiler Implementation
    • Created by Queensland University
    • LEX / FLEX (like) syntax
    • Generates C# code
    • Compatible with GPPG generated parsers
    • Easy to use
    Language framework in a managed environment February 2008 GPLEX Scanner Generator I
  • Compiler Implementation Language framework in a managed environment February 2008 GPLEX Scanner Generator II %using gppg; %option minimize %namespace Microsoft.Dynamics.PBvNext.Modelling.ExpressionCompiler /* independent scanner section start */ %x COMMENT %x STRING %x METADATA White0 [ fv] White {White0}| CmntStart /* CmntEnd */ ABStar [^* ]* ABStar2 [^"* ]* NONl [^ ]* StrStart " StrEnd "
  • Compiler Implementation Language framework in a managed environment February 2008 GPLEX Scanner Generator III %{ %} if { return (int)Tokens.KWIF; } switch { return (int)Tokens.KWSWITCH; } case { return (int)Tokens.KWCASE; } [_][a-zA-Z0-9_.]+ return (int)Tokens.IDENT; } [0-9]+ { return (int)Tokens.NUMBER; } {CmntStart}{ABStar}**{CmntEnd} { return (int)Tokens.LEX_COMMENT; } {CmntStart}{ABStar}** { BEGIN(COMMENT); return (int)Tokens.LEX_COMMENT; } <COMMENT> | <COMMENT>{ABStar}** { return (int)Tokens.LEX_COMMENT; } /* the end of the comment */ <COMMENT>{ABStar}**{CmntEnd} { BEGIN(INITIAL); return (int)Tokens.LEX_COMMENT; } /* all the other cases */ . { yyerror(&quot;illegal char&quot;); return (int)Tokens.LEX_ERROR; } %{ %}
  • Compiler Implementation
    • […] also created by Queensland University
    • YACC / Bison (like) syntax
    • […] also generates C# code
    • Compatible with GPLEX generated parsers
    • […] also easy to use
    Language framework in a managed environment February 2008 GPPG Parser Generator I
  • Compiler Implementation Language framework in a managed environment February 2008 GPPG Parser Generator II %using Microsoft.Dynamics.PBvNext.DataStructure.ExpressionAST %namespace Microsoft.Dynamics.PBvNext.Modelling.ExpressionCompiler %valuetype LexValue %partial %union { public string str; public AstNode node; } %{ public RootNode rootNode; %} %token KWIF KWSWITCH KWCASE %token STR CMT META IDENT NUMBER %left BARBAR AMPAMP %left '!' %left NEQ EQ %left '-' '+'
  • Compiler Implementation Language framework in a managed environment February 2008 GPPG Parser Generator III %% E : B { rootNode = new RootNode((ExpressionNode) $1.node); } | { rootNode = new RootNode(); } ; B : B AMPAMP B { $$.node = new BinaryBooleanExpressionNode(BooleanOperationType.AND, (ExpressionNode) $1.node, (ExpressionNode) $3.node); } ; A : A '+' A { $$.node = new BinaryArithmeticExpressionNode(ArithmeticOperationType.PLUS, (ArithmeticExpressionNode) $1.node, (ArithmeticExpressionNode) $3.node); } A : A '+' error {throw new ParsingError(”Error reducing literal expression”); } ; L : ATTR { $$.node = new AttributeNode($1.str); } | CONST { $$.node = new ConstantNode(Int32.Parse($1.str)); } | error { throw new ParsingError(”Error reducing literal expression”); } ; CONST : NUMBER { $$.str = $1.str; } ; ATTR : IDENT { $$.str = $1.str; } ;
  • Compiler Implementation Language framework in a managed environment February 2008 GPPG and GPLEX // creates a new scanner Scanner scanner = new Scanner();   // creates a new parser Parser parser = new Parser();   // sets the scanner for the parser parser.scanner = scanner;   // sets the stream to parse scanner.buffer = new Scanner.StringBuff( this .Value);   // parses the file parser.Parse();   // retrieves the output root node this .RootNode = parser.rootNode;
  • Compiler Implementation
    • One of the 23 GOF design patterns
    • Is a technique used to provide a separation between the algorithm and an object structure
    • Uses the concept of “visitor” (algorithm) and “visitable” (structure) to implement that separation
    Language framework in a managed environment February 2008 Visitor Pattern I
  • Compiler Implementation Language framework in a managed environment February 2008 Visitor Pattern II
  • Compiler Implementation
    • // implementation of the IVisitable (Element) in the class ProductNode
    • public override void Accept( IVisitor visitor)
    • {
    • // first visit
    • visitor.Visit( this );
    •  
    • if (!visitor.State)
    • return ;
    •  
    • foreach ( ProductElementNode node in productElementList)
    • {
    • node.Accept(visitor);
    • }
    •  
    • // second visit (optional)
    • visitor.Visit( this );
    • }
    Language framework in a managed environment February 2008 Visitor Pattern III
  • Compiler Implementation
    • Using visitor to provide the semantic analysis creates a simple yet powerful way of doing it
    • In case the language is complex it might be necessary to use a stack to give some memory […] a symbols table is also useful (hashing)
    • Maybe multiple visits are required (multiple pass)
    Language framework in a managed environment February 2008 Semantic Analysis I
  • Compiler Implementation
    • public void Visit( ProductNode productNode)
    • {
    • // tests if the symbol is already defined
    • if (symbols.contains(productNode.ProductValue))
    • throw new DuplicateValueException (productNode);
    • }
    • public void Visit( VariableNode variableNode)
    • {
    • // tests if the symbol is already defined in the local table
    • if (localSymbols.contains(variableNode.VariableValue))
    • throw new DuplicateValueException (variableNode);
    • if (variableNode.Array)
    • {
    • // tests if the array size is valid
    • if (variableNode.ArrayIndex < 1)
    • throw new InvalidArrayxception (variableNode);
    • }
    • }
    Language framework in a managed environment February 2008 Semantic Analysis II
  • Compiler Implementation
    • Visitor is the perfect choice in case XML is the target generated code
    • A stack is required for layout management (in case the AST is not well organized)
    • Dual visit is required, in order to close the xml nodes
    Language framework in a managed environment February 2008 Code Generation I
  • Compiler Implementation
    • public void Visit( ProductNode productNode)
    • {
    • // tests if it is the first visit
    • if (testVisit(productNode))
    • {
    • xmlTextWriter.WriteStartElement( “product&quot; );
    • xmlTextWriter.WriteAttributeString( &quot;value&quot; , productNode.ProductValue);
    • ...
    • }
    • // in case it is the second visit
    • else
    • xmlTextWriter.WriteEndElement();
    • }
    Language framework in a managed environment February 2008 Code Generation II
  • Compiler Implementation
    • Recursive descent of the XML document structure
    • This method is more hard coded and less flexible than the usage of visitors
    • Memory maybe necessary depending on the XML structure
    • Dual visit is not necessary
    Language framework in a managed environment February 2008 XML Interpretation I
  • Compiler Implementation
    • private ProductNode ParseProduct( XmlNode productNode)
    • {
    • // creates the product node
    • ProductNode node = new ProductNode ();
    •  
    • // retrieves all the attributes from the node
    • XmlAttributeCollection xmlAttributeCollection = productNode.Attributes;
    •  
    • foreach ( XmlAttribute attribute in xmlAttributeCollection)
    • {
    • // gets the name of the attribute
    • String name = attribute.Name;
    •  
    • // gets the value of the attribute
    • String value = attribute.Value;
    •  
    • if (name.Equals( &quot;value&quot; ))
    • node.ProductValue = value;
    • else
    • }
    • // gets the child nodes
    • XmlNodeList list = productNode.ChildNodes;
    •  
    • foreach ( XmlNode xmlNode in list)
    • {
    • ProductElementNode productElementNode = ParseProductElement(xmlNode);
    • node.ProductElementList.Add(productElementNode);
    • }
    •  
    • return node;
    • }
    Language framework in a managed environment February 2008 XML Interpretation II
  • Compiler Implementation
    • Syntax highlighting implementation is straightforward
    • Using the lexer is possible to associate a color with each of the tokens we want to colorize
    • The auto completion services ( “Intellisense” ) are more complex and require a good implementation of the compiler to provide a good level of usage
    Language framework in a managed environment February 2008 Compiler Services I
  • Compiler Implementation
    • Many problems emerge for the code completion
    • Parser must be able to process “all” the erroneous situations in order to be able to reduce them
    • Context localization is a very complex task
    • Performance is an issue […] incremental AST is the solution (not easy to implement)
    Language framework in a managed environment February 2008 Compiler Services II
  • Compiler Implementation
    • Another pattern of the 23 GOF design patterns
    • “ Adapts” one interface for a class into one that a client expects
    • Provides a simple system of combining a legacy implementation and a modern one
    Language framework in a managed environment February 2008 Adapter and Plug-ins I
  • Compiler Implementation Language framework in a managed environment February 2008 Adapter and Plug-ins II
  • Compiler Implementation
    • Gives a new level of abstraction
    • No implementation source code required
    • Limitation on the interface create low level of interference between both parts
    • Separation of two levels of behavior internal and external (no intrusion)
    Language framework in a managed environment February 2008 Adapter and Plug-ins III
  • Compiler Implementation
    • Combining a plug-in system and the Visitor pattern may provide a whole new level of flexibility to the compiler
    • Can be used in:
      • Dynamic targeted code generation (multiple target languages)
      • Interpretation of multiple source codes
      • Reuse of old interpreters and generators (Adapter pattern)
      • etc.
    Language framework in a managed environment February 2008 Adapter and Plug-ins IV
  • Developed Solution
    • Project developed as a thesis for Master
    • Timeline from March 2007 to June 2007
    • Small language for product configuration
    • Language developed completely in managed code
    • Complete framework for external use (André’s project)
    Language framework in a managed environment February 2008 Introduction
  • Developed Solution
    • Language for product configuration
    • Object Orientation (OO)
    • Declarative Syntax
    • Syntax highlighting and code completion services
    • Managed Environment (.NET CLR)
    • Compiler (modeling) + interpreter (configuration)
    Language framework in a managed environment February 2008 Project Description
  • Developed Solution
    • Defines the product block as the base block, equivalent to class, extension support
    • Supports the definition of the BOM and Route structures
    • The inheritance on the product variables, BOM structure, Route structure and constraints
    • Contains namespaces for context separation
    Language framework in a managed environment February 2008 The Pml Language
  • Developed Solution Language framework in a managed environment February 2008 The Modeling Environment Compiler Tools Pml Compiler Client Code Generator Lexical Analyzer Semantic Analyzer Syntactical Analyzer Pml Code Tools Syntax Highlighter Code Completer AST
  • Developed Solution
    • Semantic analyzer based in a recursive descent of the Pml AST using C# code
    • Multiple code output (XPML, XCML), but it possible to add others
    Language framework in a managed environment February 2008 The Compiler
    • The lexical and the syntactical analyzers are based respectively in the GPLEX and GPPG solutions
  • Developed Solution
    • Provide simple way to produce and compile Pml code
    • Use the current location of the caret to send information to the client
    • Rely on the context information to provide accurate completion data
    • Based in dictionaries (hash) to provide a fast and responsive system
    Language framework in a managed environment February 2008 The Compiler Tools
  • Developed Solution Language framework in a managed environment February 2008 The Configuration Environment Configuration Engine Adapter Abstraction Layer Microsoft Constraint Solver Adapter Microsoft Parallel Constraint Solver Adapter Interpretation Tools Client API Microsoft Parallel Constraint Solver Microsoft Constraint Solver AST Adapters
  • Developed Solution
    • Supports both XCML and XPML model formats
    • Uses the .NET XML parsing library (DOM based)
    • Outputs a simple AST used by the various visitors
    • Faster than interpreting Pml code directly
    Language framework in a managed environment February 2008 The Interpretation Tools
  • Developed Solution
    • Provides the necessary support to load multiple constraint solvers
    • .NET Reflection based
    • “ Uses” the Adapter pattern to load the adapters
    • Requires the Visitor pattern to adapt the AST contents to the selected solver
    • Reference implementation contains support for two Microsoft based constraint solvers
    Language framework in a managed environment February 2008 The Adapter Abstraction Layer
  • Developed Solution
    • Main entry point for the configuration process
    • Controls the interpretation of the compiled models
    • Calls the AAL for loading and running of the various constraint solvers
    • Controls the external API calls
    Language framework in a managed environment February 2008 The Configuration Engine
  • Demo Language framework in a managed environment February 2008 Demo
    • It’s really easy to construct a good compiler in a managed environment
    • The usage of design patterns in some of the compilation steps can provide a simple and clean design decision
    • Syntax highlighting is a good and easy to implement compiler tool
    • Code completion “in extremis” is a complex task and requires a good compiler design
    • Combining the Visitor and the Adapter and a plug-in system is ideal for a flexible compiler
    Conclusions Language framework in a managed environment February 2008
    • Questions
    Questions Language framework in a managed environment February 2008