Domain models are the most important asset in widely accepted software development approaches, like Domain-Driven Design (DDD), yet those models are still implicitly represented in programs. Model-Driven Engineering (MDE) regards those models as representable entities that are amenable to automated analysis and processing, facilitating quality assurance while increasing productivity in software development processes. Although this connection is not new, very few approaches facilitate adoption of MDE tooling without compromising existing value, their data. Moreover, switching to MDE tooling usually involves re-engineering core parts of an application, hindering backward compatibility and, thereby, continuous integration, while requiring an up-front investment in training in specialized modeling frameworks. In those approaches that overcome the previous problem, there is no clear indication from a quantitative point of view of the extent to which adopting state-of-the-art MDE practices and tooling is feasible or advantageous.
In this work, we advocate a code-first approach to modeling through an approach for applying MDE techniques and tools to existing object-oriented software applications that fully preserves the semantics of the original application, which need not be modified. Our approach consists both of a semi- automated method for specifying explicit view models out of existing object-oriented applications and of a conservative extension mechanism that enables the use of such view models at run time, where view model queries are resolved on demand and view model updates are propagated incrementally to the original application. This mechanism enables an iterative, flexible application of MDE tooling to software applications, where metamodels and models do not exist explicitly. An evaluation of this extension mechanism, implemented for Java applications and for view models atop the Eclipse Modeling Framework (EMF), has been conducted with an industry- targeted benchmark for decision support systems, analyzing performance and scalability of the synchronization mechanism. Backward propagation of large updates over very large views is instant.
Slides used at ASE'19. More information available at https://arturboronat.info
2. Application Domain Model (DDD)
Code-First Approach
VALUE = WORKING SOFTWARE
[Landre et al. OOPSLA’07]CONS: NOT SYSTEMATIC
DDD experts required
?
scattered knowledge
PROS: PRAGMATIC
4. Application Domain Model (MDE)
Model-First Approach (MDE) VALUE = KNOWLEDGE
PROS: SYSTEMATIC
By 2024, low-code application
development will be responsible
for more than 65% of application
development activity.
[Gartner, August 2019]
5. Application Domain Model (MDE)
Model-First Approach (MDE) VALUE = KNOWLEDGE
PROS: SYSTEMATIC
CONS: HEAVYWEIGHT
MDE experts required
[Seybold et al. SLE’16]
[Hutchinson et al. ICSE’11]
code
[Whittle et al. MODELS’13]
models more agility required
6. Combination of both? Code-First MDE
TCP-DS benchmark
SQL queries
Customers who have returned items more than 20% more
often than the average customer returns for a store in the
state TN for the year 2000
JPA
Java
OCL-like queries
Eclipse
Modeling
Framework
(EMF)model?
?
data?
7. Code-First MDE: Contributions
MDE-agnostic MDE-aware
synchronizer
Empirical evaluation:
• Overhead
• Scalability
M1
M0
system state view
app domain model view model
Synchronization model
(EMF-Syncer):
• model@runtime
• backward
compatible
• incremental
Feature-oriented
algebraic
conceptualization:
• modeling in MDE
agile sw dev
• cherry picking
Feature-Oriented
Conceptualization
Syncing Model
EMF-Syncer
Overhead and
Scalability
View update problem
[Foster et al. POPL’05]
12. View Update Problem and Our Solution
system states views
app domain model view model
Feature-Oriented
Conceptualization
SyncingModel
EMF-Syncer
Overhead
Scalability
migration
Use cases
a) View model is already given
b) View model is reified from
code
reification
View update problem
Updating system
snapshot or view creates
inconsistencies
[Foster et al. POPL’05]
13. View Update Problem and Our Solution
system states views
app domain model view model
Feature-Oriented
Conceptualization
SyncingModel
EMF-Syncer
Overhead
Scalability
syncing
policy
syncing
Syncing policy
graph isomorphism up to
feature renaming
Reversible
14. View Update Problem and Our Solution
system states views
app domain model view model
Feature-Oriented
Conceptualization
SyncingModel
EMF-Syncer
Overhead
Scalability
syncing
policy
syncing
Syncing policy
graph isomorphism up to
feature renaming
Reversible
15. EMF-Syncer
system state
app domain model view model
Feature-Oriented
Conceptualization
SyncingModel
EMF-Syncer
Overhead
Scalability
syncing
policy
view
syncingsyncing
Java EMF
Initial stage: synchronization on demand
1.Initialization of objects
16. EMF-Syncer
system state
app domain model view model
Feature-Oriented
Conceptualization
SyncingModel
EMF-Syncer
Overhead
Scalability
syncing
policy
syncing
view
Java EMF
Initial stage: synchronization on demand
1. Initialization of objects
2. On-demand synchronization feature
values
17. EMF-Syncer
system state
app domain model view model
Feature-Oriented
Conceptualization
SyncingModel
EMF-Syncer
Overhead
Scalability
syncing
policy
syncing
view
Java EMF
Initial stage: synchronization on demand
Incremental propagation stage:
1. On-demand synchronization for update
18. EMF-Syncer
system state
app domain model view model
Feature-Oriented
Conceptualization
SyncingModel
EMF-Syncer
Overhead
Scalability
syncing
policy
view
Java EMF
Initial stage: synchronization on demand
Incremental propagation stage:
1. On-demand synchronization for update
2. Update
19. EMF-Syncer
system state
app domain model view model
Feature-Oriented
Conceptualization
SyncingModel
EMF-Syncer
Overhead
Scalability
syncing
policy
Initial stage: synchronization on demand
Incremental propagation stage:
1. On-demand synchronization for update
2. Update
3. Backward propagation
Java EMF
syncing
view
20. Queries (overhead/scalability)
2 4 8 16
EMF 1st EMF 2nd
1
10
100
1000
10000
100000
0.0625 0.125 0.25 0.5 1 2 4 8 16
Time(ms.)
Size factor
Q3
SQL Java 1st Java 2nd EMF 1st EMF 2nd
Feature-Oriented
Conceptualization
SyncingModel
EMF-Syncer
Overhead
Scalability
SQL
EMF 1st
EMF 2nd
BIG DATA ANALYSIS
Time (ms.)
Size factor
TPC-DS benchmark
Java port that allows for no sexism
Experiments run on standard laptop
22. Conclusions
Reasonable overhead
Millions of objects
Models@runtime
Backward compatible
Deferred initialization
and incremental prop
Reification harnesses
modeling practices
Cherry picking tools:
gradual MDE adoption
Feature-oriented
algebraic
framework
Syncing Model
EMF-Syncer
Overhead and
Scalability 0.0001
0.0010
0.0100
0.1000
1.0000
10.0000
100.0000
1000. 0000
10000.0000
0.0625 0.125 0.25 0.5 1 2 4 8 16
Time(ms.)
Size factor
Q1
SQL Java 1st Java 2nd EMF 1st EMF 2nd
10
100
1000
10000
100000
0.0625 0.125 0.25 0.5 1 2 4 8 16
Time(ms.)
Size factor
Q2
SQL Java 1st Java 2nd EMF 1st EMF 2nd
1
10
100
1000
10000
100000
0.0625 0.125 0.25 0.5 1 2 4 8 16
Time(ms.)
Size factor
Q3
SQL Java 1st Java 2nd EMF 1st EMF 2nd
system states views
app domain model view model
syncing
policy
syncing
Code-First Model-Driven Engineering:
On the Agile Adoption of MDE
Artur BoronatASE’19. San Diego, CA, USA.
Experiments: http://bit.ly/emf-syncer-tcpds
0.01
0.1
1
10
100
1000
10000
55 84 167 333 664 1,288 2,527 5,007 9,408
Time(ms.)
Reference model size (in thousands of objects)
Baseline Q1_Initialization Q1_Backward Q2_Initialization Q3_Initialization Q3_Backward
mainstreamtechnology
MDEtechnology
Editor's Notes
Advantages
Immediate feedback on system (better communication)
Continuous delivery: no layers of indirection, product build iteratively and incrementally
The model is in the code: PLs are including modeling abstractions (classes, lambdas…)
PRAGMATIC
Drawbacks
Knowledge scattered in code: design emerges from each iteration via re-factoring
Yet programming languages are not modeling languages: less concise, less expressive
Expertise is also required for embedding a domain model in code
NOT SYSTEMATIC
VALUE = KNOWLEDGE
Advantages
Communication
Explicit models
Systematic (and automated) sw development:
SYSTEMATIC
Drawbacks
Syncing models and code
Heavyweight sw frameworks
Specialized expertise required
Hutchinson: change of mentality - progressive iterative
NOT PRACTICAL
VALUE = KNOWLEDGE
Advantages
Communication
Explicit models
Systematic (and automated) sw development:
SYSTEMATIC
Drawbacks
Syncing models and code
Heavyweight sw frameworks
Specialized expertise required
Hutchinson: change of mentality - progressive iterative
NOT PRACTICAL
VALUE = KNOWLEDGE
Advantages
Communication
Explicit models
Systematic (and automated) sw development:
SYSTEMATIC
Drawbacks
Syncing models and code
Heavyweight sw frameworks
Specialized expertise required
Hutchinson: change of mentality - progressive iterative
NOT PRACTICAL
Intro to benchmark
Modeling SDK for Visual Studio - Domain-Specific Languages
https://docs.microsoft.com/en-us/visualstudio/modeling/modeling-sdk-for-visual-studio-domain-specific-languages?view=vs-2019
Key message: what is missing
Backward-compatible at service level
EMF lock-in for implementation
Access model written manually
Backward compatibility
Models@runtime
Experiments with sizes of
1000 objects
Approaches to apply MDE tooling (Challenges to MDE tooling adoption)
Reverse engineering
Adaptation (static and run time)
Assumption
OO domain model can be extracted or projected from code
Contributions on the left
Algebraic conceptualization for transferring results from algebra to software engineering/conceptual modeling
Feature-oriented
Enriching domain models with a richer modeling language
Synchronization model
No adaptation required: metamodel-independent, focus on reuse straightaway
Ensures backward compatibility
Models@runtime: views can be obtained and updated at run-time (view model must be present)
Incremental on-demand syncing mechanism at feature level
Empirical evaluation with millions of objects: overhead and scalability
Assumptions:
OO domain model (EMF?)
Synchronization policy by name
Contributions on the right
Algebraic conceptualization for transferring results from algebra to software engineering/conceptual modeling
Feature-oriented
Synchronization model
No adaptation required: metamodel-independent, focus on reuse straightaway
Ensures backward compatibility
Models@runtime: views can be obtained and updated at run-time (view model must be present)
Incremental on-demand syncing mechanism at feature level
Empirical evaluation with millions of objects: overhead and scalability
Assumptions:
OO domain model (EMF?)
Synchronization policy by name
Conceptualization allows to enrich domain models
Feature type:f_m: c \rightarrow t
Feature value: f : o \mapsto v_i
Update: (\{\mathit{upset},\mathit{delete}\} \times f_m: c \rightarrow t), (\Delta,\cdot)
Conceptualization allows to enrich domain models
Feature type:f_m: c \rightarrow t
Feature value: f : o \mapsto v_i
Update: (\{\mathit{upset},\mathit{delete}\} \times f_m: c \rightarrow t), (\Delta,\cdot)
Conceptualization allows to enrich domain models
Feature type:f_m: c \rightarrow t
Feature value: f : o \mapsto v_i
Update: (\{\mathit{upset},\mathit{delete}\} \times f_m: c \rightarrow t), (\Delta,\cdot)
Conceptualization allows to enrich domain models
Feature type:f_m: c \rightarrow t
Feature value: f : o \mapsto v_i
Update: (\{\mathit{upset},\mathit{delete}\} \times f_m: c \rightarrow t), (\Delta,\cdot)
Contributions on the right
Algebraic conceptualization for transferring results from algebra to software engineering/conceptual modeling
Feature-oriented
Synchronization model
No adaptation required: metamodel-independent, focus on reuse straightaway
Ensures backward compatibility
Models@runtime: views can be obtained and updated at run-time (view model must be present)
Incremental on-demand syncing mechanism at feature level
Empirical evaluation with millions of objects: overhead and scalability
Assumptions:
OO domain model (EMF?)
Synchronization policy by name
Syncing policy defines consistency relation between domain model and view model
Syncing policy defines consistency relation between domain model and view model
Introduce EMF
Syncing policy defines consistency relation between domain model and view model
Introduce EMF
Syncing policy defines consistency relation between domain model and view model
Introduce EMF
Syncing policy defines consistency relation between domain model and view model
Introduce EMF
Syncing policy defines consistency relation between domain model and view model
Introduce EMF
Syncing policy defines consistency relation between domain model and view model
TPC-CS benchmark
Correctness cross-checked with SQL
Loading: SQL via JPA only returns ids, Java/EMF loading not considered (Java objects are assumed to be in memory)
Sizes, why the table StoreReturns
RQ1: Does the time overhead imposed by the synchronization algorithms, both during the initial phase and during the incremental propagation phase, hinder the pragmatism of the approach?
RQ2: Does the synchronization algorithm scale well with respect to the size of the given data set in the Java application?
Model sizes
Why is Java more efficient than SQL? contextual information is already available in memory for evaluating the query whereas SQL queries have to traverse very large tables in the database to find that initial context.
EMF-SYNCER introduces some overhead when computing queries over EMF objects when comparing the same queries over Java objects : instrumentation in EMF code performed by EMF-SYNCER
Initial syncing linear on the size of the model and propagation linear on the size of the update
Unit of data is feature values and not object (so model sizes help in comparing the approach to other benchmarks but one has to consider the density of feature values in an object)
Framework:
Cherry picking tools: facilitates creativity and gradual adoption of MDE
Experiments: millions of objectshttp://bit.ly/emf-syncer-tcpds
Model sizes
Why is Java more efficient than SQL? contextual information is already available in memory for evaluating the query whereas SQL queries have to traverse very large tables in the database to find that initial context.
EMF-SYNCER introduces some overhead when computing queries over EMF objects when comparing the same queries over Java objects : instrumentation in EMF code performed by EMF-SYNCER
DDD
Individuals and interactions over processes and toolsWorking software over comprehensive documentationCustomer collaboration over contract negotiationResponding to change over following a plan
Principles behind the Agile Manifesto
We follow these principles:Our highest priority is to satisfy the customerthrough early and continuous deliveryof valuable software.
Welcome changing requirements, even late indevelopment. Agile processes harness change forthe customer's competitive advantage.
Deliver working software frequently, from acouple of weeks to a couple of months, with apreference to the shorter timescale.
Business people and developers must worktogether daily throughout the project.
Build projects around motivated individuals.Give them the environment and support they need,and trust them to get the job done.
The most efficient and effective method ofconveying information to and within a developmentteam is face-to-face conversation.
Working software is the primary measure of progress.
Agile processes promote sustainable development.The sponsors, developers, and users should be ableto maintain a constant pace indefinitely.
Continuous attention to technical excellenceand good design enhances agility.
Simplicity--the art of maximizing the amountof work not done--is essential.
The best architectures, requirements, and designsemerge from self-organizing teams.
At regular intervals, the team reflects on howto become more effective, then tunes and adjustsits behavior accordingly.
DDD
Diagrams as communication tool, the model is the code
Needs to be visual: what is MDE? Why is it worth it?
Contributions on the right
Algebraic conceptualization for transferring results from algebra to software engineering/conceptual modeling
Feature-oriented
Synchronization model
No adaptation required: metamodel-independent, focus on reuse straightaway
Ensures backward compatibility
Models@runtime: views can be obtained and updated at run-time (view model must be present)
Incremental on-demand syncing mechanism at feature level
Empirical evaluation with millions of objects: overhead and scalability
Assumptions:
OO domain model (EMF?)
Synchronization policy by name
Contributions on the right
Algebraic conceptualization for transferring results from algebra to software engineering/conceptual modeling
Feature-oriented
Synchronization model
No adaptation required: metamodel-independent, focus on reuse straightaway
Ensures backward compatibility
Models@runtime: views can be obtained and updated at run-time (view model must be present)
Incremental on-demand syncing mechanism at feature level
Empirical evaluation with millions of objects: overhead and scalability
Contributions on the right
Algebraic conceptualization for transferring results from algebra to software engineering/conceptual modeling
Feature-oriented
Synchronization model
No adaptation required: metamodel-independent, focus on reuse straightaway
Ensures backward compatibility
Models@runtime: views can be obtained and updated at run-time (view model must be present)
Incremental on-demand syncing mechanism at feature level
Empirical evaluation with millions of objects: overhead and scalability
Assumptions:
OO domain model (EMF?)
Synchronization policy by name
Introduce EMF
Syncing policy defines consistency relation between domain model and view model
Introduce EMF
Syncing policy defines consistency relation between domain model and view model
Introduce EMF
Syncing policy defines consistency relation between domain model and view model
Introduce EMF
Syncing policy defines consistency relation between domain model and view model
Introduce EMF
Syncing policy defines consistency relation between domain model and view model
Introduce EMF
Syncing policy defines consistency relation between domain model and view model