EGL Open: Extensible Model Compilers Tim W Wilson – EGL Chief Architect
Evolution of Language Abstraction Hardware OS Runtime Platform Assembler C, COBOL Java EGL One One One One One One One Many Many Many One Single Machine Multiple Machine Future One Many Evolution of Execution Environment Evolution of Programming Languages Range of Deployment Options
EGL Basic Concepts Various component types that programmers can define Data Item  – Creates a Type alias from extended data declaration Record  - aggregate of typed fields into new type Program  - Main program starts run unit – used for batch processing Library  – Static set of data and functions – local to run-unit Service  – Remotable set of functions Interface  – Abstract definition of a Service Handler  – Object-based component containing instance data and functions ExternalType  – Abstract type definition of a type implementation that exists outside of EGL Annotations and Stereotypes Meta data applied to program elements Drives transformations as well as tooling  Abstract Statements meant to be extended get, add, delete, replace, close, call, etc.
Extensible Abstractions - Metadata Metadata allows concepts to be added without forcing specific implementations Common semantics modeled, i.e. CRUD operations on database Remote Invocation UI input validation Same semantics can apply across runtimes and frameworks Compiler/Generator maps semantic concepts to given runtime Metadata supplies the specifics
Extensible Metadata – SQLRecord Example // A Record type definition mapped to a database table Record  Order  type   SQLRecord  {  tableNames = [“ORDER”], keyItems=[“orderID”] } orderID OrderID {  columnName =“ORDERID” }; dateOrderPlaced  date  { columname=“CREATDTE” }; totalValue  money?  ; End Have you ever wondered what these actually are and where they come from?
Extensible Metadata - Annotation Types All annotations are instances of an annotation type An Annotation Type is a stereotyped Record definition Record  ColumnName  type   Annotation  {..} value  string ; end … DataItem  SSN  char (9){  columnName=“DB_SSN”, … } end
Extensible Metadata - Stereotypes Stereotypes are Annotation Types that have been annotated with the @Stereotype annotation Record  SQLRecord  type  Annotation { targets = [ ElementKind.Record ]  @Stereotype{ memberAnnotations = [Column, IsReadOnly, …] } } tableNames  string [][]; .. End
EGL Compiler as Model Compiler EGL is a programming language mixed together with a modeling mechanism and extensible model compiler Has conceptual affinity with UML Action Languages What is an Extensible Model Compiler? New concepts added to programming model e.g. Entity, UI programming model, new Datatypes (i.e. ZonedUnicodeDecimal) Existing compiler extended to map instances of new concepts used in the code to appropriate runtime Very high fidelity interleaving of new extended code generation into existing code generation Can override/extend any aspect down to low level Expressions
EDT – Code generation process Java EGL Source Code EGL Compiler Model  Extensible Code Generation Framework Java JavaScript  C/C++ .Net … Pluggable Generators Eclipse JEE Dojo Pluggable Extension Examples Possible Android COBOL CICS/IMS
Module Structure of EDT: org.eclipse.edt org.eclipse.edt.ide org.eclipse.edt.core org.eclipse.edt.gen
The EDT Modeling Framework: org.eclipse.edt.mof MOF (Meta Object Framework) Model Meta model used to create other models Used to create the EGL Model EGL Model Meta model used by compiler to create EGL elements Program, Record, Expression, etc EGL syntax can be used to define both MOF and EGL model elements EGL Compiler can create instances of either model EGL can therefore be used to define arbitrary models MOF Model EGL Model
The MOF (Meta Object Framework) Model
EDT MOF Framework All values are instances of EObject – including EClass All EObject values have a reference to their EClass instance Dynamic interface for manipulating EObjects e.g.  myEObject.eGet( “field1” ) Serialization to Binary or XML form Abstracted such that other serializations possible Replaces Intermediate Representation (IR) files seen in RBD today Java Class generator for creating specific java classes for each EClass High performance Easier programming Transformation framework Initial java class implementations of each EClass generated using this framework All EGL code generators based off same framework
EGL Meta Model
Using EGL to define a Model EGL Meta Model defined by EGL itself Special Stereotype MofClass tells compiler to create EClass instances instead of ExternalType instances Compiler creates  EClass  instance because of  MofClass  Stereotype Compiler creates normal  Externaltype  instance
XML Serialization of EClass - StructPart This object is an instance of EClass Instances of org.eclipse.edt.mof.egl.Field
XML Serialization of ExternalType - Customer This object is an instance of  org.eclipse.edt.mof.egl.ExternalType Instances of org.eclipse.edt.mof.egl.Field
Extensible Model Compilers: org.eclipse.edt.gen Core Model Transformation framework Model to Model, i.e. EDT MOF model to EMF model Model to Text, i.e. EGL Model model to Java All EDT code generation based upon this framework Simple and lightweight and highly extensible Extensibility comes from configurable ‘Templates’ A Template is simply a java Class Each model element has a Template associated with it Each Template implements methods that perform some transformation on its given model element All template method invocation follows the inheritance hierarchy of the given model If a given template method does not exist in the specific template of the model element method lookup will go up the super type chain Extending an existing transformation is simply a matter of adding and replacing templates with new ones that: Add new Replace the original Extend the original
EDT Core Generators EDT will provide ‘core’ generators for Java and JavaScript A ‘core’ generator is one that implements a transformation to a target runtime technology that adheres to the semantics of the ‘Core’ EGL Language EGL Core will be the EGL specification containing: Syntax and semantics for declaring EGL definitions Execution semantics for statements and expressions Semantic descriptions for extensible IO statements GET, ADD, etc. with no description for specifics like SQL access EGL Core will not specify any metadata specifications except for those necessary to define other metadata Annotation and Stereotype are types themselves known to the compiler  Core generators
EDT Extended Generators An Extended Generator contains: Specific set of metadata types, i.e. SQLRecord, TextProgram, etc Extension to a core generator with new templates built for new set of metadata types Runtime support libraries if necessary EDT will also provide a set of extended generators to support common platforms JEE Generator extends core Java generator Support of Web Services, JPA, etc. Core JavaScript Generator extended to handle current RUI programming model Extended generators
EGL Compilers and the IDE An EGL Compiler: Is a set of Metadata extensions with an intended programming model Compiles EGL source using those definitions into IR files Is configured with a set of extended generators that generate code to intended runtime Many EGL Compilers exist in the IDE at the same time as separate plug-ins Each ‘family’ of generators manage their inputs/outputs independently Each project configures the compiler of choice Each resource inside that project can choose a particular set of generators to use Configured Compilers: EDT, RBD, Android
RBD / EDT Integration Direction RBD 8.0.1 Eclipse EGL Tools COBOL Generator Open source IBM EGL Parser and Compiler Java Generator JavaScript Generator EDT EGL Tools EDT Java Generator EDT JavaScript Generator EGL Parser and Compiler Generation Framework RBD Future Eclipse Extended EGL Tools RBD COBOL Generator RBD Java Generator RBD JavaScript Generator EGL Tools EDT Java Generator EDT JavaScript Generator EGL Parser and Compiler Generation Framework
Android Generator: an example
UI Programming Model Example: Define Abstract Concepts Abstract concepts derived from concepts common across UI frameworks Window – deployable UI unit Android – Activity Web – HTML Page View – Super type of all UI controls Android – View SWT – Widget ViewGroup – contains View children Android – ViewGroup Dojo – Container TableLayout – kind of ViewGroup for Tables Window View ViewGroup TableLayout 1 contentView * children
UI Programming Model Example: Define Concepts in EGL Abstract concepts defined using  Externaltype with  AbstractType  stereotype Event handlers modeled as Delegates Inheritance modeled directly Externaltype  declarations allow for full Class modeling
UI Programming Model Example: Define Concepts in EGL Core UI Concepts Stereotypes for implementation Concrete Widgets
Example Usage: Mortgage Calculator Event handler declaration Stereotype used to implement a Window Similar conceptually to RUIHandler Invocation of the Amortize Service Declaration of Widgets and properties
Mapping to Target Platform: Android and Dojo Abstract concepts need concrete mapping to intended runtime platform Native Android is a Java framework Android generator would be an extension of the core EDT Java generator Dojo is a JavaScript framework Dojo generator would be an extension of the core EDT JavaScript generator This example shows a simple mapping to Android platform Use of widgets and layout Calling a web service This example is simply meant to show what it means to build an EGL compiler extension – not an intention to do native Android as a strategic direction
Create the AndroidGenerator class Android generator extends core Java generator Adds command line parameters as necessary Define where the configuration is for adding and overriding code generation templates Define where the configuration is for mapping abstract types to concrete classes in the runtime
Create Necessary Templates Override existing method from Template from Core Template method invocation
Map Model to Android Templates Mapping of model classes to their templates
Running Mortgage Calculator on Android Emulator
Demo
EGL open Tools – Xtool and Xref Lars Kaare Andersen Xact Consulting A/S CTO and Partner
Agenda for this presentation Background for the tools and support to the EGL open The main  purpose  of Xtool and Xref Xtool main functions Xref main functions Future directions Implement/integrate with new version of the EGL open Requests from customers Integration with RTC  Integration with RAA
Background for the tools and support to the EGL open Xact have supported the EGL open project from the start. Added a large number of requirements and API’s needed by customer or Xact. Provided test cases for IBM development team. Executed test based on the early versions of the EGL open software. Goals  To support the general goal to turn EGL into a open platform. To be able to create tool to support EGL development, management and change. Customer or Business Partners like Xact. A large number of tool to support EGL will make EGL better  choice for new and existing customers.
Background for the tools Xact have daily contact with large EGL customers. Customers that have developed EGL code over the last 10-20 years. Migrated/converted EGL from other languages. Common situation is that the customers have a large number EGL code lines. The reuse of EGL part are on a very high level. (very positive). Common Data dictionary Common records/data areas for program, functions, services …. Common logging Common error handling …… Thousands of common packages and programs
EGL code relations – direct and indirect use of parts Program2 Program1 Record2 Program3 Program4 Function1 Function2 Function3 Function4 Record4 Record3 Record1
EGL program relations and usages Test case 1 Test case 2 Test case 3 Sub System xx Program1 Program4 Program2 Program3 Program5 Program6 Program7 Program10 Program8 Program9 Program11 Program12 Program13 Program16 Program14 Program15 Program17 Program18
Xtool and Xref Xtool Version management – repository adaptors.  Build management. Creation of dynamic package groups. Release and Change management. Xref Register, store and find cross references in all the EGL code – direct and indirect RBD + Xtool and Xref plugins Xtool Build server Xtool Database server
Version Control related to States (Test, Systemtest, Production) Limited to:  Latest or State Version in State
Xtool Build Server Xtool Build server Load package versions from State Build EGL code Update State information Update Xref information Manage Build Progress Manage Change control
Xref – Search for parts
Xref Information - Call Stack
Xref – Search for References
Xtool - Synchronize
Xtool – Main package View
Thank You

EGL Conference 2011 - EGL Open

  • 1.
    EGL Open: ExtensibleModel Compilers Tim W Wilson – EGL Chief Architect
  • 2.
    Evolution of LanguageAbstraction Hardware OS Runtime Platform Assembler C, COBOL Java EGL One One One One One One One Many Many Many One Single Machine Multiple Machine Future One Many Evolution of Execution Environment Evolution of Programming Languages Range of Deployment Options
  • 3.
    EGL Basic ConceptsVarious component types that programmers can define Data Item – Creates a Type alias from extended data declaration Record - aggregate of typed fields into new type Program - Main program starts run unit – used for batch processing Library – Static set of data and functions – local to run-unit Service – Remotable set of functions Interface – Abstract definition of a Service Handler – Object-based component containing instance data and functions ExternalType – Abstract type definition of a type implementation that exists outside of EGL Annotations and Stereotypes Meta data applied to program elements Drives transformations as well as tooling Abstract Statements meant to be extended get, add, delete, replace, close, call, etc.
  • 4.
    Extensible Abstractions -Metadata Metadata allows concepts to be added without forcing specific implementations Common semantics modeled, i.e. CRUD operations on database Remote Invocation UI input validation Same semantics can apply across runtimes and frameworks Compiler/Generator maps semantic concepts to given runtime Metadata supplies the specifics
  • 5.
    Extensible Metadata –SQLRecord Example // A Record type definition mapped to a database table Record Order type SQLRecord { tableNames = [“ORDER”], keyItems=[“orderID”] } orderID OrderID { columnName =“ORDERID” }; dateOrderPlaced date { columname=“CREATDTE” }; totalValue money? ; End Have you ever wondered what these actually are and where they come from?
  • 6.
    Extensible Metadata -Annotation Types All annotations are instances of an annotation type An Annotation Type is a stereotyped Record definition Record ColumnName type Annotation {..} value string ; end … DataItem SSN char (9){ columnName=“DB_SSN”, … } end
  • 7.
    Extensible Metadata -Stereotypes Stereotypes are Annotation Types that have been annotated with the @Stereotype annotation Record SQLRecord type Annotation { targets = [ ElementKind.Record ] @Stereotype{ memberAnnotations = [Column, IsReadOnly, …] } } tableNames string [][]; .. End
  • 8.
    EGL Compiler asModel Compiler EGL is a programming language mixed together with a modeling mechanism and extensible model compiler Has conceptual affinity with UML Action Languages What is an Extensible Model Compiler? New concepts added to programming model e.g. Entity, UI programming model, new Datatypes (i.e. ZonedUnicodeDecimal) Existing compiler extended to map instances of new concepts used in the code to appropriate runtime Very high fidelity interleaving of new extended code generation into existing code generation Can override/extend any aspect down to low level Expressions
  • 9.
    EDT – Codegeneration process Java EGL Source Code EGL Compiler Model Extensible Code Generation Framework Java JavaScript C/C++ .Net … Pluggable Generators Eclipse JEE Dojo Pluggable Extension Examples Possible Android COBOL CICS/IMS
  • 10.
    Module Structure ofEDT: org.eclipse.edt org.eclipse.edt.ide org.eclipse.edt.core org.eclipse.edt.gen
  • 11.
    The EDT ModelingFramework: org.eclipse.edt.mof MOF (Meta Object Framework) Model Meta model used to create other models Used to create the EGL Model EGL Model Meta model used by compiler to create EGL elements Program, Record, Expression, etc EGL syntax can be used to define both MOF and EGL model elements EGL Compiler can create instances of either model EGL can therefore be used to define arbitrary models MOF Model EGL Model
  • 12.
    The MOF (MetaObject Framework) Model
  • 13.
    EDT MOF FrameworkAll values are instances of EObject – including EClass All EObject values have a reference to their EClass instance Dynamic interface for manipulating EObjects e.g. myEObject.eGet( “field1” ) Serialization to Binary or XML form Abstracted such that other serializations possible Replaces Intermediate Representation (IR) files seen in RBD today Java Class generator for creating specific java classes for each EClass High performance Easier programming Transformation framework Initial java class implementations of each EClass generated using this framework All EGL code generators based off same framework
  • 14.
  • 15.
    Using EGL todefine a Model EGL Meta Model defined by EGL itself Special Stereotype MofClass tells compiler to create EClass instances instead of ExternalType instances Compiler creates EClass instance because of MofClass Stereotype Compiler creates normal Externaltype instance
  • 16.
    XML Serialization ofEClass - StructPart This object is an instance of EClass Instances of org.eclipse.edt.mof.egl.Field
  • 17.
    XML Serialization ofExternalType - Customer This object is an instance of org.eclipse.edt.mof.egl.ExternalType Instances of org.eclipse.edt.mof.egl.Field
  • 18.
    Extensible Model Compilers:org.eclipse.edt.gen Core Model Transformation framework Model to Model, i.e. EDT MOF model to EMF model Model to Text, i.e. EGL Model model to Java All EDT code generation based upon this framework Simple and lightweight and highly extensible Extensibility comes from configurable ‘Templates’ A Template is simply a java Class Each model element has a Template associated with it Each Template implements methods that perform some transformation on its given model element All template method invocation follows the inheritance hierarchy of the given model If a given template method does not exist in the specific template of the model element method lookup will go up the super type chain Extending an existing transformation is simply a matter of adding and replacing templates with new ones that: Add new Replace the original Extend the original
  • 19.
    EDT Core GeneratorsEDT will provide ‘core’ generators for Java and JavaScript A ‘core’ generator is one that implements a transformation to a target runtime technology that adheres to the semantics of the ‘Core’ EGL Language EGL Core will be the EGL specification containing: Syntax and semantics for declaring EGL definitions Execution semantics for statements and expressions Semantic descriptions for extensible IO statements GET, ADD, etc. with no description for specifics like SQL access EGL Core will not specify any metadata specifications except for those necessary to define other metadata Annotation and Stereotype are types themselves known to the compiler Core generators
  • 20.
    EDT Extended GeneratorsAn Extended Generator contains: Specific set of metadata types, i.e. SQLRecord, TextProgram, etc Extension to a core generator with new templates built for new set of metadata types Runtime support libraries if necessary EDT will also provide a set of extended generators to support common platforms JEE Generator extends core Java generator Support of Web Services, JPA, etc. Core JavaScript Generator extended to handle current RUI programming model Extended generators
  • 21.
    EGL Compilers andthe IDE An EGL Compiler: Is a set of Metadata extensions with an intended programming model Compiles EGL source using those definitions into IR files Is configured with a set of extended generators that generate code to intended runtime Many EGL Compilers exist in the IDE at the same time as separate plug-ins Each ‘family’ of generators manage their inputs/outputs independently Each project configures the compiler of choice Each resource inside that project can choose a particular set of generators to use Configured Compilers: EDT, RBD, Android
  • 22.
    RBD / EDTIntegration Direction RBD 8.0.1 Eclipse EGL Tools COBOL Generator Open source IBM EGL Parser and Compiler Java Generator JavaScript Generator EDT EGL Tools EDT Java Generator EDT JavaScript Generator EGL Parser and Compiler Generation Framework RBD Future Eclipse Extended EGL Tools RBD COBOL Generator RBD Java Generator RBD JavaScript Generator EGL Tools EDT Java Generator EDT JavaScript Generator EGL Parser and Compiler Generation Framework
  • 23.
  • 24.
    UI Programming ModelExample: Define Abstract Concepts Abstract concepts derived from concepts common across UI frameworks Window – deployable UI unit Android – Activity Web – HTML Page View – Super type of all UI controls Android – View SWT – Widget ViewGroup – contains View children Android – ViewGroup Dojo – Container TableLayout – kind of ViewGroup for Tables Window View ViewGroup TableLayout 1 contentView * children
  • 25.
    UI Programming ModelExample: Define Concepts in EGL Abstract concepts defined using Externaltype with AbstractType stereotype Event handlers modeled as Delegates Inheritance modeled directly Externaltype declarations allow for full Class modeling
  • 26.
    UI Programming ModelExample: Define Concepts in EGL Core UI Concepts Stereotypes for implementation Concrete Widgets
  • 27.
    Example Usage: MortgageCalculator Event handler declaration Stereotype used to implement a Window Similar conceptually to RUIHandler Invocation of the Amortize Service Declaration of Widgets and properties
  • 28.
    Mapping to TargetPlatform: Android and Dojo Abstract concepts need concrete mapping to intended runtime platform Native Android is a Java framework Android generator would be an extension of the core EDT Java generator Dojo is a JavaScript framework Dojo generator would be an extension of the core EDT JavaScript generator This example shows a simple mapping to Android platform Use of widgets and layout Calling a web service This example is simply meant to show what it means to build an EGL compiler extension – not an intention to do native Android as a strategic direction
  • 29.
    Create the AndroidGeneratorclass Android generator extends core Java generator Adds command line parameters as necessary Define where the configuration is for adding and overriding code generation templates Define where the configuration is for mapping abstract types to concrete classes in the runtime
  • 30.
    Create Necessary TemplatesOverride existing method from Template from Core Template method invocation
  • 31.
    Map Model toAndroid Templates Mapping of model classes to their templates
  • 32.
    Running Mortgage Calculatoron Android Emulator
  • 33.
  • 34.
    EGL open Tools– Xtool and Xref Lars Kaare Andersen Xact Consulting A/S CTO and Partner
  • 35.
    Agenda for thispresentation Background for the tools and support to the EGL open The main purpose of Xtool and Xref Xtool main functions Xref main functions Future directions Implement/integrate with new version of the EGL open Requests from customers Integration with RTC Integration with RAA
  • 36.
    Background for thetools and support to the EGL open Xact have supported the EGL open project from the start. Added a large number of requirements and API’s needed by customer or Xact. Provided test cases for IBM development team. Executed test based on the early versions of the EGL open software. Goals To support the general goal to turn EGL into a open platform. To be able to create tool to support EGL development, management and change. Customer or Business Partners like Xact. A large number of tool to support EGL will make EGL better choice for new and existing customers.
  • 37.
    Background for thetools Xact have daily contact with large EGL customers. Customers that have developed EGL code over the last 10-20 years. Migrated/converted EGL from other languages. Common situation is that the customers have a large number EGL code lines. The reuse of EGL part are on a very high level. (very positive). Common Data dictionary Common records/data areas for program, functions, services …. Common logging Common error handling …… Thousands of common packages and programs
  • 38.
    EGL code relations– direct and indirect use of parts Program2 Program1 Record2 Program3 Program4 Function1 Function2 Function3 Function4 Record4 Record3 Record1
  • 39.
    EGL program relationsand usages Test case 1 Test case 2 Test case 3 Sub System xx Program1 Program4 Program2 Program3 Program5 Program6 Program7 Program10 Program8 Program9 Program11 Program12 Program13 Program16 Program14 Program15 Program17 Program18
  • 40.
    Xtool and XrefXtool Version management – repository adaptors. Build management. Creation of dynamic package groups. Release and Change management. Xref Register, store and find cross references in all the EGL code – direct and indirect RBD + Xtool and Xref plugins Xtool Build server Xtool Database server
  • 41.
    Version Control relatedto States (Test, Systemtest, Production) Limited to: Latest or State Version in State
  • 42.
    Xtool Build ServerXtool Build server Load package versions from State Build EGL code Update State information Update Xref information Manage Build Progress Manage Change control
  • 43.
    Xref – Searchfor parts
  • 44.
  • 45.
    Xref – Searchfor References
  • 46.
  • 47.
    Xtool – Mainpackage View
  • 48.