Softeam is a French company with more than 20 years of experience producing UML-based modelling environments on top of its Modelio CASE tool. With the increase of complexity of models and modelling teams, came the demand for supporting scalable collaborative tools for modelling. This new demand lead to the creation of Constellation, Softeam’s enterprise model management solution. As part of this effort, Softeam joined the MONDO FP7 EU project, which develops tools and methodologies for dealing with the challenges of scalability in model driven engineering. In this experimentation, we will describe the integration of the model indexer Hawk produced by the Enterprise Systems research group at the University of York into the commercial modelling tool Modelio and into its Constellation collaboration tools, produced by Softeam. The focus of this presentation will be on the technical difficulties of integrating with commercial production grade tools, and on a benchmark of the performance of this integration. In summary, Hawk integration was performed by Softeam engineers; a functional prototype was obtained in three months, and this prototype was subsequently improved until the end of the project. On the benchmarking side, for big models, we found out that in some situations Hawk index requires half the space required by Modelio to store models, and that combining Hawk and EGL generates documents between two and three orders of magnitude faster than Modelio itself.
Integrating research grade model indexing technologies to commercial modelling tools: feedback and benchmarks
1. Integrating research grade
model indexing technologies to
commercial modelling tools:
feedback and benchmarks
Marcos Almeida, Antonin Abhervé, Alessandra
Bagnato
SOFTEAM – France
Antonio García-Domínguez, Konstantinos Barmpis
UoY– UK
May 2016 – Paris - ICSSEA 1
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
2. Modelio for Software
and System Engineering
• UML editor with 20 years’ history
• SysML
• MARTE
• UTP
• Code generation
• Documentation
• Teamwork
• Available under open source
at Modelio.org!
May 2016 – Paris - ICSSEA 2
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
3. The MONDO Project
MONDO is a STREP FP7 EU project
Start: 11/2013 End: 4/2016
Total cost: 3.7M€
Challenges:
Model management languages struggle with
models containing more than a few 100Ks
model elements
XMI is great for interoperability but its
performance is poor
There is little guidance on designing large
DSLs / DSLs for large models
May 2016 – Paris - ICSSEA 3
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
4. Partner roles
Use Cases, requirements
validation
Ikerlan (ES)
Softeam (FR)
Soft-Maint (FR)
UNINOVA (PT)
Dissemination and
industry standards
Open Group (UK)
Technology providers
Softeam (FR)
UNINOVA (PT)
Research/development
ARMINES (FR)
Auton. Univ of Madrid (ES)
Budapest University of Technology and
Economics (HU)
Univ of York (UK)
May 2016 – Paris - ICSSEA 4
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
5. May 2016 – Paris - ICSSEA 5
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
From a single model to a constellation of
models
Enterprise level project
management
Project catalog
Fragments
organization
Inter-projects links
Versions and variants
Communication
– Reports generation
– Project dashboard
– News and activity feeds
Shared model
repository
– SVN Model Fragment
repository
– RAMC Model
Fragment repository
– HTTP Model Fragment
repository
Constellation main features
6. Videos on Modelio &
Constellation
May 2016 – Paris - ICSSEA 6
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
7. Goals
Before Mondo
Constellation knows
model fragments by
their names.
We did not have
efficient tools
dedicated to querying
Models.
After Mondo
We looked for a way
to know the content
and organization of all
elements in model
fragments.
We looked for a way
to query all of our
models.
May 2016 – Paris - ICSSEA 7
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
8. Objectives
Evaluate MONDO technologies within
Modelio
Supporting large and complex model
repositories (sets of models)
Supporting large collaborating teams
Should:
Implement a demonstrator
Document experiences gained
May 2016 – Paris - ICSSEA 8
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
9. Measures to evaluate
Capability of MONDO to provide a user
friendly interface for managing
ModelToModel transformations
ModelToText transformations.
Capability of MONDO to provide scalable
execution of transformation information.
Capability of MONDO to provide scalable
execution of queries.
Capability of MONDO to provide an
improved collaborative environment.
May 2016 – Paris - ICSSEA 9
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
10. Hawk
A heterogeneous model indexing
framework.
May 2016 – Paris - ICSSEA 10
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
11. Hawk - Constellation Integration
Integrated architecture
May 2016 – Paris - ICSSEA 11
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
12. Hawk - Constellation : Results
Integration of Hawk query engine to
constellation.
Capability to execute
queries on all of our models
Provide an holistic view of
the composition of our
repositories
May 2016 – Paris - ICSSEA 12
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
13. Hawk - Constellation : Results
Introducing Statistics based on all our
models of repositories.
Implemented using specific Hawk
queries
May 2016 – Paris - ICSSEA 13
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
14. Hawk - Constellation : Results
Capability to provide an improved
collaborative environment.
Ex : Help customer to compose a project in
constellation using Hawk query to filter
available fragments,…
May 2016 – Paris - ICSSEA 14
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
15. Video on Constellation
integration
May 2016 – Paris - ICSSEA 15
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
16. Benchmarks
Hawk space use & indexing time
Model to Text transformations
May 2016 – Paris - ICSSEA 16
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
18. Benchmarks: Hawk space use &
indexing time – 1/2
Model repository
3.44 GB / 1,239,829 model elements
1M model elements of many different types
Modelio cannot load it
Generated models
From 1K elements to 1M elements (most typical
types: Class, Packages, Operations etc.)
Environment
Machine: Dual core 2.7 GHz / 8GB RAM Dell
Notebook
• This is more about trends than absolute numbers!
Modelio 3.4.1b
Hawk 1.0.0.201602181354
May 2016 – Paris - ICSSEA 18
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
19. Benchmarks: Hawk space use &
indexing time – 2/2
Questions
Is storage space and indexing time linear?
• How does it compare to Modelio storage
space linearity?
How long does it take to index huge
models?
May 2016 – Paris - ICSSEA 19
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
20. Results – Generated models - Trends
Disk space grows linearly (good sign!)
Growth is less steep than Modelio’s
Hawk tends to require less space for
very big models.
May 2016 – Paris - ICSSEA 20
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
21. Results – Generated models - Trends
Indexing time grows linearly (good sign!)
Even if indexing may take a lot of time,
re-indexing is quite fast.
On a later version, indexing is 25x faster!
May 2016 – Paris - ICSSEA 21
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
22. Results – Model repository
Indexing time
4 days and 2 hours
• Remember: Modelio can’t handle all these
models at the same time!!
Update time
26 min
• Still good for back-end tasks, like
computing stats, generating docs, etc.
May 2016 – Paris - ICSSEA 22
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
23. Results – Model repository – on a
later version
Indexing time
5 hours 14 minutes (yes, 18x faster!)
• Helped optimizing Hawk for models
composed of lots of small files.
Update time
50 s
• Quite good for collaborative scenarios!
May 2016 – Paris - ICSSEA 23
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
24. Model to Text transformations
Document generation task (Markdown)
Implemented in
MONDO: EGL (Epislon Generation
Language)
Modelio: Jython
May 2016 – Paris - ICSSEA 24
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
25. Results – M2T
Here’s where the use of MONDO
technologies pays the most
Hawk + EGL 175 to 602 times faster than
Modelio
~3.3h
~19s
May 2016 – Paris - ICSSEA 25
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
26. Video on Hawk indexing and
Model to text
transformations
May 2016 – Paris - ICSSEA 26
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
27. Specific measures – Full compliance
Measure Conclusions
Time improvement for change
propagation and notification among
concurrent users
Re-indexing time under 5ms
Time improvement percentage on
query execution
Queries on document generation
from 6 to 700 times faster than
Modelio Desktop
Time improvement percentage on the
execution of transformations for text
generation
Document generation from 6 to
700 times faster than Modelio
Desktop
Time improvement percentage on the
execution of transformations for
model generation
● Generation 2-17 times faster
than Desktop Modelio on a
well configured server and
moderately large models
● Provides functionality that was
not available in Constellation
before
May 2016 – Paris - ICSSEA 27
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
28. Conclusion
Hawk provides us a way to index all
our model fragment whatever their
hosting technology.
Hawk provides a powerful query
engine which allow us to know the
content of our model fragment on
Constellation side.
Packaged as JAR, the integration of
Hawk to our commercial tool was easy.
May 2016 – Paris - ICSSEA 28
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
29. Further enhancements
Improvements
MONDO collaboration tools should support
other modelling technologies besides EMF.
Future plans
MEASURE collaboration
CloudATL (model to model transformations
tool) integration to Constellation
May 2016 – Paris - ICSSEA 29
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks
30. Questions ?
May 2016 – Paris - ICSSEA 30
Integrating research grade model indexing technologies to
commercial modelling tools: feedback and benchmarks