This presentation is tcase study about “Transforming a 15 year old model-driven application from C++ to Java”
Let me first give an overview of my presentation.
First I will introduce myself. Then I will describe the assignment that this presentation is about Since the assignment was essentially a renovation of an existing application, I will describe how renovation of old applications is done in general. Next I will describe how we applied different strategies in succession as we learned on the job. Finally I will try to draw some lessons from the experience and mention a number of problems that we did not yet find a good solution for.
What I learned in those 27 years: - Automated procedures as important as programming skills or architecture - Automated build - Version management - Automated deployment - (Automatic unit testing) - Communication and trust within team Specialties: - Software factories 25+ years - used to be new but now more common - Model-driven code generation 10+ years - Agile Development - eXtreme Programming 10+ years - Scrum 3+ years
What was our assignment?
A very succesful mortgage application - 185 screens (or parts of screens) - 100 users - regular quarterly releases - sizeable model: 200.000 lines, 10Mb of XMI Using a proprietary model-driven C++ generator written in C++ Several interfaces to external systems, queues Several native Windows DLL's for complicated calculations
Implicit metamodel (as far as known) Model in relational tables (DB2) C++ “Class Tree” to load model in memory - analogous to the EMF generated Java API Loader containing the mapping knowledge Hand-written C++ framework - persistence support classes, including lazy loading - user interface support classes Action Language: COOL (Common Object Oriented Language), used everywhere: - methods of objects - actions on UI elements (buttons, fields)
Scarce expertise - model-driven techniques in general - this particular modeler/generator in particular - C++ Dependent on outdated C++ compiler and libraries Modeler/generator maintained but not further developed: dead end Difficult to offshore Wanted: a regular Java webapplication No code generator - dependency on proprietary tool - Current developers where quite happy with the tool and productive
Start to establish goals for a new application Collect requirements Make specification Implement However: in many cases there is only source code, and if there is more documentation, it is often outdated. How to renovate: - automatic source translation? No idiomatic Java. - Better: reverse engineer specifications and re-implement - Even better (but more difficult): reverse engineer reqs. - End to Goals.
But in our case we had specification in the form of an executable (therefore correct) model. 1) Write a new C++ generator to generate Java - Problem: large conceptual gap to bridge: - C++ to Java (fluency in both needed) - Different architecture - On top of model driven skills 2) Use Mod4J as intermediate model - Smaller conceptual gap - Built-in Java architecture - Based on written reference architecture 3) Migrate model to EMF for productivity - Better tools - Better software factory 4) Mod4J incompatibly architecture - fat client ↔ layering - COOL language everywhere - lazy loading
Want to use EMF - Need an ecore metamodel - Derive from “Class Tree” - Via Enterprise Architect im/export - Generate an export program - Generate our new application - Loading time: 1,5 second - But initially 25 seconds (!) - Need a HashMap for id's
An intermediate model provides a natural place for names
Classic client server → modern three tiered layered Lazy loading assumed in de Action Language One Scope with UI scripts and Class Methods
Know your tools - EMF is great, but you need the book - Xtext2 for the COOL Parser difficult to integrate - Xtend2, nice but buggy and slow
Eclipse is difficult - Working combination of (versions of) plugins - Hard to get rid of the red crosses Continuous integration - Maven ↔ OSGi: two worlds - Solve a similar problem - Different audiences: tool/application builders - But MDCD overlap the two - Eclipse artifacts not automatically in maven repositories.
How to maintain the migrated model: - Use EMF tree editor - already quite good - Derive an Xtext grammar from the metamodel - beautify - DSL for “free” - The language is still the same - Familiar for the maintainers
Transforming a 15 year old model-driven application from C++ to Java
1Transforming a 15 year old model-drivenapplication from C++ to JavaEric Jan Malotaux Code Generation 2012, Cambridge, UK 2012-03-28
3 OVERVIEW Who I Am The Assignment Renovating applications The Strategy Lessons Learned Unsolved Problems
4 WHO AM I Eric Jan Malotaux Software Architect with Ordina, ICT service provider in the Netherlands Trained as a musician and musicologist 27 years experience in software development What I learned Automated procedures (software factory) as important as programming skills Communication and trust within a team even more important Specialties – anything that speeds up software development Software factories – automating as much as possible Agile development (extreme programming, Scrum) Model-driven code generation
The AssignmentTransforming a 15 year old model-driven application from C++ to Java
6 THE ASSIGNMENT THE APPLICATION A very succesful mortgage application 185 screens 100 users Quarterly releases Sizeable model: 200,000 lines (10Mb) of XMI Using a proprietary model-driven code generator Several interfaces to external systems: queues Several native Windows DLLs for complicated calculations
8 THE ASSIGNMENT THE MODELER/GENERATOR Old, but very complete Model in relational tables (DB2) Modeler written in MS-Access C++ Class Tree to keep model in memory Loader containing mapping (DB → Class Tree) knowledge Hand-written C++ framework, modeled Persistence support classes, including lazy loading User Interface support classes Action Language: COOL (Common Object Oriented Language) Methods on objects Actions on UI elements, like buttons Implicit metamodel (as far as known)
9 THE ASSIGNMENT WHY TRANSFORM? Dependent on outdated C++ compiler and libraries Scarce expertise Model-driven techniques in general The proprietary modeler/generator in particular C++ Modeler/generator maintained, but not further development Doesnt fit in Enterprise Architecture Difficult to offshore Want a mainstream Java web application And no code generator anymore please, thank you very much. No dependency on proprietary tools Current developers where quite happy, though
Legacy application Renovated application Goals Requirements FAST FAST Mod4J FAST model C++ C++ C++ extractor model Xtend2 mogram model extractor extractor generator (database) (xmi) (textual DSL) (xmi) Xtend2 Xpand C++ generator C++ generator generator C++ Java implementatio implementatio n n
Old Intermediate New (Enterprise Architect) SQL C++ UML EcoreM3 expressed in Abstract expressed in expressed in expressed in Syntax FAST FAST EA C++ FAST EA ecoreM2 metamodel “Class Tree” import metamodel export Expo metamodel (relational) rt (ecore) gene ra to r expressed in expressed in expressed in (xten d FMA 2) gen FMA FMA e FMA Model FMA exporM1 model gen load e object t r model (database) graph a (xmi) r (C++) t a o t r o s r (xte s n (C+ d + 2 FMA FMAM0 user data ) ) application
Ext r a Enterpri se c Legenda t Architect o given r FAST gen FAST manual FAST e EMF “Class metamo toolsmetamo r generato Tree del del a r ” (ecore) generated t o r FMA (xte n FMA FAST model Load/ d model model FM(databas extract 2 (XMI) Java API A e) ) gen e r a FAST t FMA editor o model r (xte n d JNA FMA 2 FMA ) adapters example (Java) (Java) (Java) FAST FAST maven/framewo framewo javac/ rk rk build (C++) (Java)External External (MQ) (MQ) FMA clas clas (JWS/jar FMA ses ses ) data (C++) (Java)
18 LESSONS LEARNED The value of an explicit metamodel Help in understanding the model Generated export program Generated tree editor Generated model API Basis for a new DSL
19 LESSONS LEARNED Generator architecture Consistent naming Use an intermediate model
20 LESSONS LEARNED Incompatible source/target architecture Layering: classic client → server, modern three-tiered Lazy loading assumed in the COOL Action Language One Action Language for UI and Class Methods
21 LESSONS LEARNED Know your tools Know your tools EMF is a great tool, once you obtain a copy of the EMF book Xtext2, needed for COOL Parser, difficult to integrate Xtend2, nice but still buggy
22 REMAINING UNSOLVED PROBLEMS Eclipse is a hard to tame beast! Working combination of (versions of) plugins Hard to get rid of red crosses Continuous integration Maven ↔ OSGi: two worlds Eclipse artifacts (not) in public maven repositories
23 REMAINING CHALLENGE Maintaining the migrated model Use the EMF-generated tree editor already as good as, or better than, the Access application Derive an Xtext grammar from the metamodel Manual adjustments Have a DSL almost for free Familiar concepts: the metamodel (language) was not changed.