DSLs and tool support for building  Tailored Code Generators Pedro J. Molina, PhD. Software Engineer Capgemini Spain | Valencia [email_address] http:// pjmolina.com/metalevel
Contents Context Some problems How to address it? A solution proposal Demo time Scenarios Usages Conclusions ©MMX Pedro J.Molina, PhD.
Initial Context Custom Software Development Specific developments for custom needs Staff specialised with diverse and partial  degree of knowledge: People with knowledge in the sector, in specific customers needs enterprise architects, analysts People with knowledge in specific stacks of technology SW architects, developers ©MMX Pedro J.Molina, PhD.
Main goal Understand business needs Recommend bests technologies Align IT with Business needs ©MMX Pedro J.Molina, PhD.
Problem #1 : IT Diversity Customers have diverse IT standards Preferred technology Frameworks & libraries Coding conventions and style guides Regulations to fulfil How we can target as much as possible and still have quality, reuse and be able to apply best practices? ©MMX Pedro J.Molina, PhD.
Problem #2:  Technology changes Companies needs to follow adoption paths soon or later Lack of support Obsolete technology Lack of qualified professionals Cost of maintenance (TCO) How we can reduce our dependency on technology? Make business know-how independent of the technology (as much as you can) ©MMX Pedro J.Molina, PhD.
Problem #3:  Time to Market and Costs Frequently customers avoid new developments based on about previous experiences: It will not be on time It will cost too much MDSD can turn projects feasible that were not viable with a traditional development ©MMX Pedro J.Molina, PhD.
How to address it? Enforce a strong separation between  Business Know-How & Technological Know-How Promote job specialization Automate as much as possible Better Quality and Time to Market ©MMX Pedro J.Molina, PhD.
A solution proposal Use MDSD:  Model your Business Know-How Standardize your Technological substrate to be used Generate your apps from models to your selected architecture Refine and iterate whenever Business changes or Technology changes ©MMX Pedro J.Molina, PhD.
Approaches to create code generators From 1998 till 2010 I played with lots of approaches: C++ with string concatenations XML & XSL-T ( horrible !) C++ with custom text template engine (2 engines) JSP or ASP languages  C# with Velocity C# with T4 PHP and Smarty C# or Java with StringTemplate Frameworks for building code generators Declarative Code Generation... ©MMX Pedro J.Molina, PhD.
MVC Model-View-Controller Useful pattern for Presentation Layers Crucial for  maintainability  in Code Generators StringTemplate  enforces MVC Model View Controller ©MMX Pedro J.Molina, PhD.
Conceptual Map for  Code Generation Metamodel Model Templates Code Transformations Higher  Abstraction Level  Lower Instances  Types Code Generator ©MMX Pedro J.Molina, PhD.
Conceptual Map for CG Specification Transformation Algorithms Programs Models Classes /  Types Instances ©MMX Pedro J.Molina, PhD. Source Code Template Meta-model Reverse Engineering Abstraction Synthesis Mappings Instanciation Abstraction
DSLs Domain Specific Languages to define: Metamodels Models Templates Model to Text Transformations (M2T) Model to Model Transformations (M2M) ©MMX Pedro J.Molina, PhD.
Essential Essential  is an  IDE and a runtime for creating code generators Declarative, Clean Fast Promotes reuse Forward engineering approach ©MMX Pedro J.Molina, PhD.
Motivations for a new tool Easy to use MDD tool Learning curve: as lower as possible Explore ways  to create  better and faster  code generation Declaratively describe the bricks of MDD Fast Code Generator prototyper Explore the concept of  Fowler Language Workbench IDE for creating code generators Apply  SoC  to MDD Teach the ropes of Code Generation Searching for the build bricks of MDD And enjoy in the Quest! ©MMX Pedro J.Molina, PhD.
It should be easier ©MMX Pedro J.Molina, PhD.
Metamodel Sample model Describes email Sender And recipients namespace  Meta { class   MailMessage {   Person  From;   List<Person>  To;   string  Subject;   string  Message; } class   Person { string  FullName;   string  Email; } } ©MMX Pedro J.Molina, PhD.
Models Instantiation of Metamodels Object Properties References Namespaces Sample Create mail and persons using  Meta; namespace  Model { MailMessage   FirstMessage {   From  =  Peter ;   To  = [ Alicia ,  Peter ];   Subject  = &quot;Welcomed&quot;;   Message  = &quot;Hello world!&quot;; } Person   Peter {   FullName  = &quot;Peter Mills&quot;;   Email  = &quot;pmills@acme.com&quot;; } Person   Alicia {   FullName  = &quot;Alicia Lopez&quot;;   Email  = &quot;alopez@acme.com&quot;; } } ©MMX Pedro J.Molina, PhD.
Templates StringTemplate Clean and reusable template No control inside Support for: Subtemplates Rendering Clean iteration Control language agnostic group   Templates ; genMail( msg )   ::= << From: $ msg.From : genRecipient()$ To:  $ msg.To :genRecipient() ; separator=&quot;, &quot;$ Subject: $ msg.Subject $ MESSAGE: ------------------------------ $ msg.Message $ ------------------------------ >> genRecipient ()::= << $ it.FullName $ <$ it.Email $> >> ©MMX Pedro J.Molina, PhD.
Control and Transformations Custom Control Language called Trx to: Select Model parts (Model Query) Apply templates Organize generation output files / models using Meta; using Model; namespace  Gen { transformation  Main() {   MailMessage  msg =  Model.FirstMessage ;   Templates.genMail (msg) > &quot;message.txt&quot;; } } ©MMX Pedro J.Molina, PhD.
Tool support Grammars tools Parsers construction Editor Syntax colorization Code completion Model Checker ©MMX Pedro J.Molina, PhD.
Why another tool? Fast changes to metamodel  UML tools: fixed metamodel! EMF/GMF/MS DSL Tools MVC approach to Templates Avoiding JSP approaches to code gen Test a full declarative code generator in the way I like it ©MMX Pedro J.Molina, PhD.
Composable Models Partial model definition Extensible Link them before transformation Promotes Maintainability Separation of Concerns ©MMX Pedro J.Molina, PhD.
Model transformations M2T Model to Template (StringTemplate) M2M Model 2 Model transformations Based in rules ©MMX Pedro J.Molina, PhD.
Essential Demo Hello world  Mail Sample Class Model to DB Schema Model  DB Schema Model to SQL Application Code Generation ©MMX Pedro J.Molina, PhD.
Generation Settings Just another Model! Describes the design choices and configuration options selected for the final solution ©MMX Pedro J.Molina, PhD.
Usage Scenarios I Direct Generation Benefits Forward engineering Automation Quality assurance Model Generation Settings Code  Generator Code ©MMX Pedro J.Molina, PhD.
Usage Scenarios II Addressing Multi-target code C#, Java, Ruby, PHP, C/C++, HTML + Javascript,  SQL dialects, etc. Based in generation settings selections Model Generation Settings Code Generator Code B Code A ©MMX Pedro J.Molina, PhD.
Usage Scenarios III Derive a Design Model (PIM    PSM) Benefits Technology agnostic / Vendor independence Detailed design Opportunity to tune the design if needed Abstract Model (PIM) Model  Transformation Design Model (PSM) ... ©MMX Pedro J.Molina, PhD.
Usage Scenarios IV (Semi)? Automatic Reverse Engineering Benefits Legacy apps migration Pragmatic Cost/Benefit approach Model Extraction Tool Legacy Asset ©MMX Pedro J.Molina, PhD.
Usage Scenarios V Model Migration Benefits Automatic conversion of models when metamodel changes Model A M2M Converter Model A’ Metamodel (v1) Metamodel (v2) Conforms  with Conforms  with Metamodel Evolution ©MMX Pedro J.Molina, PhD.
Usage Scenarios VI Transformation Chaining Benefits Componentization Modular transformations Testability Reuse Model M2M Design Model M2T M2T Persistence Code Logic Code M2T Documentation ©MMX Pedro J.Molina, PhD.
Usage Scenarios VII Selecting best-suitable technologies Benefits Architecture evolution Selection of best architecture based in the nature of your expected (or real) workload Measure performance Model M2T A M2T B A Implementation B Implementation Load Test Perfor-mance Measure ©MMX Pedro J.Molina, PhD.
Usage Scenarios VIII Enforcing Naming Rules Benefits Enforce company naming rules Assure consistency Central point of change for naming rules Model Naming Code M2T Named  Design Model Naming Policy Central point of change ©MMX Pedro J.Molina, PhD.
Usage Scenarios IX Assuring Compliance Accessibility, Auditing, Privacy Regulations, Security Benefits Enforce compliance Proved by design Avoid manual errors Certify that  all code  derived from this method satisfies the required compliance level Model Compliant Code Generator Compliant  Code ©MMX Pedro J.Molina, PhD.
Usage Scenarios X Technology Forward Escape Benefits Language / Architecture Evolution Leverage Legacy Code Consolidation in enterprise defined standards platforms Reduce the investment in outdated technology ASAP (Reduce TCO) Early adoption:  Faster Time to Market Opportunity to be the first one (Pioneer)! Model Old Code Generator New Code Generator Obsolete Unsupported Technology Code Latest Super cool Technology Code Extraction Tool Legacy Code ©MMX Pedro J.Molina, PhD.
Road Map for Essential Release an early  Community Edition Get  feedback  from the CG Community Command line  support (on going) Improve  M2M  transformations Include  derivations  (calculated model properties) Add model  query DSL Translate transformations to C# or Java code Pack code generators for specific needs Plugin  and extensibility model for the IDE Support it all! Projectional  editor ©MMX Pedro J.Molina, PhD.
 
Thank you! More info about  Essential  on: The Metalevel http://pjmolina.com/metalevel http:// pjmolina.com/metalevel/essential ©MMX Pedro J.Molina, PhD.

CG2010 Tailored Code Generators

  • 1.
    DSLs and toolsupport for building Tailored Code Generators Pedro J. Molina, PhD. Software Engineer Capgemini Spain | Valencia [email_address] http:// pjmolina.com/metalevel
  • 2.
    Contents Context Someproblems How to address it? A solution proposal Demo time Scenarios Usages Conclusions ©MMX Pedro J.Molina, PhD.
  • 3.
    Initial Context CustomSoftware Development Specific developments for custom needs Staff specialised with diverse and partial degree of knowledge: People with knowledge in the sector, in specific customers needs enterprise architects, analysts People with knowledge in specific stacks of technology SW architects, developers ©MMX Pedro J.Molina, PhD.
  • 4.
    Main goal Understandbusiness needs Recommend bests technologies Align IT with Business needs ©MMX Pedro J.Molina, PhD.
  • 5.
    Problem #1 :IT Diversity Customers have diverse IT standards Preferred technology Frameworks & libraries Coding conventions and style guides Regulations to fulfil How we can target as much as possible and still have quality, reuse and be able to apply best practices? ©MMX Pedro J.Molina, PhD.
  • 6.
    Problem #2: Technology changes Companies needs to follow adoption paths soon or later Lack of support Obsolete technology Lack of qualified professionals Cost of maintenance (TCO) How we can reduce our dependency on technology? Make business know-how independent of the technology (as much as you can) ©MMX Pedro J.Molina, PhD.
  • 7.
    Problem #3: Time to Market and Costs Frequently customers avoid new developments based on about previous experiences: It will not be on time It will cost too much MDSD can turn projects feasible that were not viable with a traditional development ©MMX Pedro J.Molina, PhD.
  • 8.
    How to addressit? Enforce a strong separation between Business Know-How & Technological Know-How Promote job specialization Automate as much as possible Better Quality and Time to Market ©MMX Pedro J.Molina, PhD.
  • 9.
    A solution proposalUse MDSD: Model your Business Know-How Standardize your Technological substrate to be used Generate your apps from models to your selected architecture Refine and iterate whenever Business changes or Technology changes ©MMX Pedro J.Molina, PhD.
  • 10.
    Approaches to createcode generators From 1998 till 2010 I played with lots of approaches: C++ with string concatenations XML & XSL-T ( horrible !) C++ with custom text template engine (2 engines) JSP or ASP languages C# with Velocity C# with T4 PHP and Smarty C# or Java with StringTemplate Frameworks for building code generators Declarative Code Generation... ©MMX Pedro J.Molina, PhD.
  • 11.
    MVC Model-View-Controller Usefulpattern for Presentation Layers Crucial for maintainability in Code Generators StringTemplate enforces MVC Model View Controller ©MMX Pedro J.Molina, PhD.
  • 12.
    Conceptual Map for Code Generation Metamodel Model Templates Code Transformations Higher Abstraction Level Lower Instances Types Code Generator ©MMX Pedro J.Molina, PhD.
  • 13.
    Conceptual Map forCG Specification Transformation Algorithms Programs Models Classes / Types Instances ©MMX Pedro J.Molina, PhD. Source Code Template Meta-model Reverse Engineering Abstraction Synthesis Mappings Instanciation Abstraction
  • 14.
    DSLs Domain SpecificLanguages to define: Metamodels Models Templates Model to Text Transformations (M2T) Model to Model Transformations (M2M) ©MMX Pedro J.Molina, PhD.
  • 15.
    Essential Essential is an IDE and a runtime for creating code generators Declarative, Clean Fast Promotes reuse Forward engineering approach ©MMX Pedro J.Molina, PhD.
  • 16.
    Motivations for anew tool Easy to use MDD tool Learning curve: as lower as possible Explore ways to create better and faster code generation Declaratively describe the bricks of MDD Fast Code Generator prototyper Explore the concept of Fowler Language Workbench IDE for creating code generators Apply SoC to MDD Teach the ropes of Code Generation Searching for the build bricks of MDD And enjoy in the Quest! ©MMX Pedro J.Molina, PhD.
  • 17.
    It should beeasier ©MMX Pedro J.Molina, PhD.
  • 18.
    Metamodel Sample modelDescribes email Sender And recipients namespace Meta { class MailMessage { Person From; List<Person> To; string Subject; string Message; } class Person { string FullName; string Email; } } ©MMX Pedro J.Molina, PhD.
  • 19.
    Models Instantiation ofMetamodels Object Properties References Namespaces Sample Create mail and persons using Meta; namespace Model { MailMessage FirstMessage { From = Peter ; To = [ Alicia , Peter ]; Subject = &quot;Welcomed&quot;; Message = &quot;Hello world!&quot;; } Person Peter { FullName = &quot;Peter Mills&quot;; Email = &quot;pmills@acme.com&quot;; } Person Alicia { FullName = &quot;Alicia Lopez&quot;; Email = &quot;alopez@acme.com&quot;; } } ©MMX Pedro J.Molina, PhD.
  • 20.
    Templates StringTemplate Cleanand reusable template No control inside Support for: Subtemplates Rendering Clean iteration Control language agnostic group Templates ; genMail( msg ) ::= << From: $ msg.From : genRecipient()$ To: $ msg.To :genRecipient() ; separator=&quot;, &quot;$ Subject: $ msg.Subject $ MESSAGE: ------------------------------ $ msg.Message $ ------------------------------ >> genRecipient ()::= << $ it.FullName $ <$ it.Email $> >> ©MMX Pedro J.Molina, PhD.
  • 21.
    Control and TransformationsCustom Control Language called Trx to: Select Model parts (Model Query) Apply templates Organize generation output files / models using Meta; using Model; namespace Gen { transformation Main() { MailMessage msg = Model.FirstMessage ; Templates.genMail (msg) > &quot;message.txt&quot;; } } ©MMX Pedro J.Molina, PhD.
  • 22.
    Tool support Grammarstools Parsers construction Editor Syntax colorization Code completion Model Checker ©MMX Pedro J.Molina, PhD.
  • 23.
    Why another tool?Fast changes to metamodel UML tools: fixed metamodel! EMF/GMF/MS DSL Tools MVC approach to Templates Avoiding JSP approaches to code gen Test a full declarative code generator in the way I like it ©MMX Pedro J.Molina, PhD.
  • 24.
    Composable Models Partialmodel definition Extensible Link them before transformation Promotes Maintainability Separation of Concerns ©MMX Pedro J.Molina, PhD.
  • 25.
    Model transformations M2TModel to Template (StringTemplate) M2M Model 2 Model transformations Based in rules ©MMX Pedro J.Molina, PhD.
  • 26.
    Essential Demo Helloworld Mail Sample Class Model to DB Schema Model DB Schema Model to SQL Application Code Generation ©MMX Pedro J.Molina, PhD.
  • 27.
    Generation Settings Justanother Model! Describes the design choices and configuration options selected for the final solution ©MMX Pedro J.Molina, PhD.
  • 28.
    Usage Scenarios IDirect Generation Benefits Forward engineering Automation Quality assurance Model Generation Settings Code Generator Code ©MMX Pedro J.Molina, PhD.
  • 29.
    Usage Scenarios IIAddressing Multi-target code C#, Java, Ruby, PHP, C/C++, HTML + Javascript, SQL dialects, etc. Based in generation settings selections Model Generation Settings Code Generator Code B Code A ©MMX Pedro J.Molina, PhD.
  • 30.
    Usage Scenarios IIIDerive a Design Model (PIM  PSM) Benefits Technology agnostic / Vendor independence Detailed design Opportunity to tune the design if needed Abstract Model (PIM) Model Transformation Design Model (PSM) ... ©MMX Pedro J.Molina, PhD.
  • 31.
    Usage Scenarios IV(Semi)? Automatic Reverse Engineering Benefits Legacy apps migration Pragmatic Cost/Benefit approach Model Extraction Tool Legacy Asset ©MMX Pedro J.Molina, PhD.
  • 32.
    Usage Scenarios VModel Migration Benefits Automatic conversion of models when metamodel changes Model A M2M Converter Model A’ Metamodel (v1) Metamodel (v2) Conforms with Conforms with Metamodel Evolution ©MMX Pedro J.Molina, PhD.
  • 33.
    Usage Scenarios VITransformation Chaining Benefits Componentization Modular transformations Testability Reuse Model M2M Design Model M2T M2T Persistence Code Logic Code M2T Documentation ©MMX Pedro J.Molina, PhD.
  • 34.
    Usage Scenarios VIISelecting best-suitable technologies Benefits Architecture evolution Selection of best architecture based in the nature of your expected (or real) workload Measure performance Model M2T A M2T B A Implementation B Implementation Load Test Perfor-mance Measure ©MMX Pedro J.Molina, PhD.
  • 35.
    Usage Scenarios VIIIEnforcing Naming Rules Benefits Enforce company naming rules Assure consistency Central point of change for naming rules Model Naming Code M2T Named Design Model Naming Policy Central point of change ©MMX Pedro J.Molina, PhD.
  • 36.
    Usage Scenarios IXAssuring Compliance Accessibility, Auditing, Privacy Regulations, Security Benefits Enforce compliance Proved by design Avoid manual errors Certify that all code derived from this method satisfies the required compliance level Model Compliant Code Generator Compliant Code ©MMX Pedro J.Molina, PhD.
  • 37.
    Usage Scenarios XTechnology Forward Escape Benefits Language / Architecture Evolution Leverage Legacy Code Consolidation in enterprise defined standards platforms Reduce the investment in outdated technology ASAP (Reduce TCO) Early adoption: Faster Time to Market Opportunity to be the first one (Pioneer)! Model Old Code Generator New Code Generator Obsolete Unsupported Technology Code Latest Super cool Technology Code Extraction Tool Legacy Code ©MMX Pedro J.Molina, PhD.
  • 38.
    Road Map forEssential Release an early Community Edition Get feedback from the CG Community Command line support (on going) Improve M2M transformations Include derivations (calculated model properties) Add model query DSL Translate transformations to C# or Java code Pack code generators for specific needs Plugin and extensibility model for the IDE Support it all! Projectional editor ©MMX Pedro J.Molina, PhD.
  • 39.
  • 40.
    Thank you! Moreinfo about Essential on: The Metalevel http://pjmolina.com/metalevel http:// pjmolina.com/metalevel/essential ©MMX Pedro J.Molina, PhD.