Intro Integration Evaluation Conclusions
Integration of a Graph-Based Model Indexer in
Commercial Modelling Tools
Antonio ...
Intro Integration Evaluation Conclusions
Outline
1 Intro
2 Integration
3 Evaluation
4 Conclusions
A García-Domínguez et al...
Intro Integration Evaluation Conclusions
Motivation for this work
MONDO EU project: produced Hawk indexer
Goal: tackle sca...
Intro Integration Evaluation Conclusions
Motivation for this work
MONDO EU project: produced Hawk indexer
Goal: tackle sca...
Modelio: SOFTEAM’s modelling tool
Open source, scriptable, UML/BPMN/SysML...
Structure: workspace → projects → fragments (...
Intro Integration Evaluation Conclusions
Constellation: SOFTEAM’s collab server
Admin server coordinates
agents and provid...
Intro Integration Evaluation Conclusions
Hawk: NoSQL model indexer
We go from these model files...
A García-Domínguez et al...
Intro Integration Evaluation Conclusions
Hawk: NoSQL model indexer
... to these NoSQL graphs.
A García-Domínguez et al. In...
Intro Integration Evaluation Conclusions
Hawk: NoSQL model indexer
EPackages → metamodel nodes
EClasses → type nodes
A Gar...
Intro Integration Evaluation Conclusions
Hawk: NoSQL model indexer
.xmi → file nodes
EObjects → element nodes
A García-Domí...
Intro Integration Evaluation Conclusions
Hawk: NoSQL model indexer
MM index: EPackage URI → metamodel node
File index: rep...
Intro Integration Evaluation Conclusions
Outline
1 Intro
2 Integration
3 Evaluation
4 Conclusions
A García-Domínguez et al...
Intro Integration Evaluation Conclusions
Integration approach
Key detail in Hawk: plugin-based architecture
Hawk has abstr...
Intro Integration Evaluation Conclusions
Initial legal issues
Problem
Hawk is EPL, Modelio is GPLv3: incompatible licenses...
Intro Integration Evaluation Conclusions
Initial legal issues
Problem
Hawk is EPL, Modelio is GPLv3: incompatible licenses...
.exml format
<EXT object="Component" version="3">
<DEPS>
<ID name="Component" mc="Class" uid="2d7b..."/>
<EXTID name="Elem...
.exml format: version, dependencies
<EXT object="Component" version="3">
<DEPS>
<ID name="Component" mc="Class" uid="2d7b....
.exml format: version, dependencies
<EXT object="Component" version="3">
<DEPS>
<ID name="Component" mc="Class" uid="2d7b....
.exml format: root object
<EXT object="Component" version="3">
<DEPS>
<ID name="Component" mc="Class" uid="2d7b..."/>
<EXT...
.exml format: root object
<EXT object="Component" version="3">
<DEPS>
<ID name="Component" mc="Class" uid="2d7b..."/>
<EXT...
.exml format: attributes and dependencies
<EXT object="Component" version="3">
<DEPS>
<ID name="Component" mc="Class" uid=...
.exml format: attributes and dependencies
<EXT object="Component" version="3">
<DEPS>
<ID name="Component" mc="Class" uid=...
.exml format: nested objects
<EXT object="Component" version="3">
<DEPS>
<ID name="Component" mc="Class" uid="2d7b..."/>
<...
Intro Integration Evaluation Conclusions
Hawk: tweaks needed for Modelio
References are UID-based
Parsed objects/refs can ...
Intro Integration Evaluation Conclusions
Outline
1 Intro
2 Integration
3 Evaluation
4 Conclusions
A García-Domínguez et al...
Intro Integration Evaluation Conclusions
Indexing costs on real projects (Neo4j)
File type .exml .ramc
Minimum 229B 23kB
M...
Embedding into Constellation: advanced queries
Required new OrientDB backend (Neo4j is GPLv3-licensed)
SOFTEAM embedded Ha...
Embedding into Constellation: dashboards
SOFTEAM used Hawk queries to feed project dashboards
Regular users do not need to...
Intro Integration Evaluation Conclusions
M2T with Hawk: motivation
Business case
M2T in Modelio was done through Jython sc...
Intro Integration Evaluation Conclusions
M2T with Hawk: inputs
Script size param m = 1 m = 2 m = 4 m = 8 m = 16
Project si...
Intro Integration Evaluation Conclusions
M2T with Hawk: indexing times
1 2 3 4 5 6
·105
200
400
600
800
1,000
1,200
Projec...
Intro Integration Evaluation Conclusions
M2T with Hawk: generation times
1 2 3 4 5 6
·105
100
101
102
103
Project size (mo...
Intro Integration Evaluation Conclusions
M2T with Hawk: tradeoff
1 2 3 4 5 6
·105
0
5
10
15
20
1
Project size (model elemen...
Intro Integration Evaluation Conclusions
Outline
1 Intro
2 Integration
3 Evaluation
4 Conclusions
A García-Domínguez et al...
Intro Integration Evaluation Conclusions
Conclusions and future work
Obtained results
SOFTEAM wanted querying for their mo...
End of the presentation
Questions?
@antoniogado
A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Co...
Upcoming SlideShare
Loading in …5
×

MoDELS'16 presentation: Integration of a Graph-Based Model Indexer in Commercial Modelling Tools

346 views

Published on

In this MoDELS'16 paper, we show how the Hawk model indexer was extended to index Modelio models, and how it was embedded into the Constellation collaboration server from SOFTEAM to allow for efficient querying and metric extraction from its web-based interface.

Published in: Software
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
346
On SlideShare
0
From Embeds
0
Number of Embeds
21
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

MoDELS'16 presentation: Integration of a Graph-Based Model Indexer in Commercial Modelling Tools

  1. 1. Intro Integration Evaluation Conclusions Integration of a Graph-Based Model Indexer in Commercial Modelling Tools Antonio García-Domínguez, Konstantinos Barmpis, Dimitrios S. Kolovos, Marcos Aurelio Almeida da Silva, Antonin Abherve and Alessandra Bagnato Aston University, University of York, SOFTEAM Cadextan MoDELS’16 October 7th, 2016 A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 1 / 27
  2. 2. Intro Integration Evaluation Conclusions Outline 1 Intro 2 Integration 3 Evaluation 4 Conclusions A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 2 / 27
  3. 3. Intro Integration Evaluation Conclusions Motivation for this work MONDO EU project: produced Hawk indexer Goal: tackle scalability of MDE in its various facets One product was Hawk, a scalable model indexer: https://github.com/mondo-project/mondo-hawk Hawk mirrors fragmented models into NoSQL graphs SOFTEAM: Constellation collab server needed search SOFTEAM has 20+ years experience on UML tooling Modelio is their open-source modelling tool Joined MONDO due to scalability challenges Created Constellation: Modelio collab server Constellation had no search box – need indexing! A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 3 / 27
  4. 4. Intro Integration Evaluation Conclusions Motivation for this work MONDO EU project: produced Hawk indexer Goal: tackle scalability of MDE in its various facets One product was Hawk, a scalable model indexer: https://github.com/mondo-project/mondo-hawk Hawk mirrors fragmented models into NoSQL graphs SOFTEAM: Constellation collab server needed search SOFTEAM has 20+ years experience on UML tooling Modelio is their open-source modelling tool Joined MONDO due to scalability challenges Created Constellation: Modelio collab server Constellation had no search box – need indexing! A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 3 / 27
  5. 5. Modelio: SOFTEAM’s modelling tool Open source, scriptable, UML/BPMN/SysML... Structure: workspace → projects → fragments (.exml) Projects can import .ramc libraries (zipped fragments)
  6. 6. Intro Integration Evaluation Conclusions Constellation: SOFTEAM’s collab server Admin server coordinates agents and provides web UI Agent nodes (HTTP/SVN) access/host fragments No querying facilities A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 5 / 27
  7. 7. Intro Integration Evaluation Conclusions Hawk: NoSQL model indexer We go from these model files... A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 6 / 27
  8. 8. Intro Integration Evaluation Conclusions Hawk: NoSQL model indexer ... to these NoSQL graphs. A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 6 / 27
  9. 9. Intro Integration Evaluation Conclusions Hawk: NoSQL model indexer EPackages → metamodel nodes EClasses → type nodes A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 6 / 27
  10. 10. Intro Integration Evaluation Conclusions Hawk: NoSQL model indexer .xmi → file nodes EObjects → element nodes A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 6 / 27
  11. 11. Intro Integration Evaluation Conclusions Hawk: NoSQL model indexer MM index: EPackage URI → metamodel node File index: repo + file path → file node Users can define custom indices by attribute/expression A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 6 / 27
  12. 12. Intro Integration Evaluation Conclusions Outline 1 Intro 2 Integration 3 Evaluation 4 Conclusions A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 7 / 27
  13. 13. Intro Integration Evaluation Conclusions Integration approach Key detail in Hawk: plugin-based architecture Hawk has abstraction layers for: Database backends (Neo4j, OrientDB) Model persistence formats (XMI, UML, Modelio...) Query languages (Epsilon Object Language) Steps taken 1 Expose Modelio metamodels to Hawk 2 Write .exml/.ramc parser 3 Extend Hawk with Modelio specifics 4 Embed into Constellation A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 8 / 27
  14. 14. Intro Integration Evaluation Conclusions Initial legal issues Problem Hawk is EPL, Modelio is GPLv3: incompatible licenses Cannot reuse neither metamodel code nor parsing code Solutions SOFTEAM wrote M2T transformation to expose metamodels as Apache-licensed library (ModelioMetamodelLib) http://github.com/aabherve/modelio-metamodel-lib Hawk includes transformation to Ecore Hawk devs collaborated with SOFTEAM to write new parsers Clean-room approach: no looking at Modelio code! A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 9 / 27
  15. 15. Intro Integration Evaluation Conclusions Initial legal issues Problem Hawk is EPL, Modelio is GPLv3: incompatible licenses Cannot reuse neither metamodel code nor parsing code Solutions SOFTEAM wrote M2T transformation to expose metamodels as Apache-licensed library (ModelioMetamodelLib) http://github.com/aabherve/modelio-metamodel-lib Hawk includes transformation to Ecore Hawk devs collaborated with SOFTEAM to write new parsers Clean-room approach: no looking at Modelio code! A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 9 / 27
  16. 16. .exml format <EXT object="Component" version="3"> <DEPS> <ID name="Component" mc="Class" uid="2d7b..."/> <EXTID name="Element" mc="Class" uid="4ed7..."/> </DEPS> <OBJECT> <ID name="Component" mc="Class" uid="2d7b..."/> <PID name="Architecture" mc="Package" uid="ea87..."/> <ATTRIBUTES> <ATT name="Name">Component</ATT>... </ATTRIBUTES> <DEPENDENCIES> <COMP relation="OwnedOperation">...</COMP> <COMP relation="Parent">...</COMP> </DEPENDENCIES> </OBJECT> </EXT>
  17. 17. .exml format: version, dependencies <EXT object="Component" version="3"> <DEPS> <ID name="Component" mc="Class" uid="2d7b..."/> <EXTID name="Element" mc="Class" uid="4ed7..."/> </DEPS> <OBJECT> <ID name="Component" mc="Class" uid="2d7b..."/> <PID name="Architecture" mc="Package" uid="ea87..."/> <ATTRIBUTES> <ATT name="Name">Component</ATT>... </ATTRIBUTES> <DEPENDENCIES> <COMP relation="OwnedOperation">...</COMP> <COMP relation="Parent">...</COMP> </DEPENDENCIES> </OBJECT> </EXT> Format version Dependencies
  18. 18. .exml format: version, dependencies <EXT object="Component" version="3"> <DEPS> <ID name="Component" mc="Class" uid="2d7b..."/> <EXTID name="Element" mc="Class" uid="4ed7..."/> </DEPS> <OBJECT> <ID name="Component" mc="Class" uid="2d7b..."/> <PID name="Architecture" mc="Package" uid="ea87..."/> <ATTRIBUTES> <ATT name="Name">Component</ATT>... </ATTRIBUTES> <DEPENDENCIES> <COMP relation="OwnedOperation">...</COMP> <COMP relation="Parent">...</COMP> </DEPENDENCIES> </OBJECT> </EXT> Format version Dependencies
  19. 19. .exml format: root object <EXT object="Component" version="3"> <DEPS> <ID name="Component" mc="Class" uid="2d7b..."/> <EXTID name="Element" mc="Class" uid="4ed7..."/> </DEPS> <OBJECT> <ID name="Component" mc="Class" uid="2d7b..."/> <PID name="Architecture" mc="Package" uid="ea87..."/> <ATTRIBUTES> <ATT name="Name">Component</ATT>... </ATTRIBUTES> <DEPENDENCIES> <COMP relation="OwnedOperation">...</COMP> <COMP relation="Parent">...</COMP> </DEPENDENCIES> </OBJECT> </EXT> Object name, metaclass, UID Same UID = same object Happens when importing libraries Container UID EMF compatibility would require deriving containments
  20. 20. .exml format: root object <EXT object="Component" version="3"> <DEPS> <ID name="Component" mc="Class" uid="2d7b..."/> <EXTID name="Element" mc="Class" uid="4ed7..."/> </DEPS> <OBJECT> <ID name="Component" mc="Class" uid="2d7b..."/> <PID name="Architecture" mc="Package" uid="ea87..."/> <ATTRIBUTES> <ATT name="Name">Component</ATT>... </ATTRIBUTES> <DEPENDENCIES> <COMP relation="OwnedOperation">...</COMP> <COMP relation="Parent">...</COMP> </DEPENDENCIES> </OBJECT> </EXT> Object name, metaclass, UID Same UID = same object Happens when importing libraries Container UID EMF compatibility would require deriving containments
  21. 21. .exml format: attributes and dependencies <EXT object="Component" version="3"> <DEPS> <ID name="Component" mc="Class" uid="2d7b..."/> <EXTID name="Element" mc="Class" uid="4ed7..."/> </DEPS> <OBJECT> <ID name="Component" mc="Class" uid="2d7b..."/> <PID name="Architecture" mc="Package" uid="ea87..."/> <ATTRIBUTES> <ATT name="Name">Component</ATT>... </ATTRIBUTES> <DEPENDENCIES> <COMP relation="OwnedOperation">...</COMP> <COMP relation="Parent">...</COMP> </DEPENDENCIES> </OBJECT> </EXT> Attributes: name-value pairs Dependencies: references COMPositions (containment?) LINKs (non-containment)
  22. 22. .exml format: attributes and dependencies <EXT object="Component" version="3"> <DEPS> <ID name="Component" mc="Class" uid="2d7b..."/> <EXTID name="Element" mc="Class" uid="4ed7..."/> </DEPS> <OBJECT> <ID name="Component" mc="Class" uid="2d7b..."/> <PID name="Architecture" mc="Package" uid="ea87..."/> <ATTRIBUTES> <ATT name="Name">Component</ATT>... </ATTRIBUTES> <DEPENDENCIES> <COMP relation="OwnedOperation">...</COMP> <COMP relation="Parent">...</COMP> </DEPENDENCIES> </OBJECT> </EXT> Attributes: name-value pairs Dependencies: references COMPositions (containment?) LINKs (non-containment)
  23. 23. .exml format: nested objects <EXT object="Component" version="3"> <DEPS> <ID name="Component" mc="Class" uid="2d7b..."/> <EXTID name="Element" mc="Class" uid="4ed7..."/> </DEPS> <OBJECT> <ID name="Component" mc="Class" uid="2d7b..."/> <PID name="Architecture" mc="Package" uid="ea87..."/> <ATTRIBUTES> <ATT name="Name">Component</ATT>... </ATTRIBUTES> <DEPENDENCIES> <COMP relation="OwnedOperation">...</COMP> <COMP relation="Parent">...</COMP> </DEPENDENCIES> </OBJECT> </EXT> <OBJECT> <ID name="..." mc="Generalization" uid="58e6..."/> <PID name="Component" mc="Class" uid="2d7b..."/> <ATTRIBUTES>...</ATTRIBUTES> <DEPENDENCIES> <LINK relation="SuperType"> <ID name="Element" mc="Class" uid="4ed7..."/> </LINK> </DEPENDENCIES> </OBJECT> PID here is XML container’s UID: not always the case! LINK to object in another .exml
  24. 24. Intro Integration Evaluation Conclusions Hawk: tweaks needed for Modelio References are UID-based Parsed objects/refs can declare they are UID-based If so, Hawk will maintain/use UID → object index Same UID = same object Parsed resources can declare they provide “singletons” If so, Hawk will merge objects with same UID into 1 node A single element node can belong then to multiple files Container-first rather than containment-first Optional automated derivation of EMF-style containments Only needed for compatibility with EMF-based tools Not needed for Epsilon compatibility A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 15 / 27
  25. 25. Intro Integration Evaluation Conclusions Outline 1 Intro 2 Integration 3 Evaluation 4 Conclusions A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 16 / 27
  26. 26. Intro Integration Evaluation Conclusions Indexing costs on real projects (Neo4j) File type .exml .ramc Minimum 229B 23kB Median 1.51kB 8.45MB Maximum 2.77MB 30.98MB Count 201 526 23 Stages 2GB 4GB 8GB Frag ins. 2554s 2502s 2371s Frag conn. 728s 541s 443s Cont deriv. 1851s 1234s 1123s Total 5171s 4287s 3958s Inputs SOFTEAM provided 15 in-house projects 2GB between .exml and .ramc, 1.23M elems Memory consumption W/containment: 2GB+ Without: 1GB (3146s) Diminishing returns A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 17 / 27
  27. 27. Embedding into Constellation: advanced queries Required new OrientDB backend (Neo4j is GPLv3-licensed) SOFTEAM embedded Hawk into web UI in 3 months First result: EOL query console
  28. 28. Embedding into Constellation: dashboards SOFTEAM used Hawk queries to feed project dashboards Regular users do not need to know any queries
  29. 29. Intro Integration Evaluation Conclusions M2T with Hawk: motivation Business case M2T in Modelio was done through Jython scripting SOFTEAM wanted M2T agents in Constellation Code generation as a service Which is more efficient? Modelio + Jython? Hawk + Epsilon Generation Language? Need to take into account indexing times as well! A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 20 / 27
  30. 30. Intro Integration Evaluation Conclusions M2T with Hawk: inputs Script size param m = 1 m = 2 m = 4 m = 8 m = 16 Project size 80MB 150MB 279MB 814MB 1.8GB .exml size 58MB 104MB 179MB 422MB 752MB .exml count 1 255 2 087 3 430 7 849 13 790 Element count 47 981 88 451 154 281 367 840 657 228 SOFTEAM wrote simple UML model generator in Jython Size parameter controls number of UML classes and number of attributes/references per class A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 21 / 27
  31. 31. Intro Integration Evaluation Conclusions M2T with Hawk: indexing times 1 2 3 4 5 6 ·105 200 400 600 800 1,000 1,200 Project size (model elements) Indexingtime(s) 500 1,000 1,500 2,000 2,500 OrientDBdiskspaceusage(MB) Time Disk space A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 22 / 27
  32. 32. Intro Integration Evaluation Conclusions M2T with Hawk: generation times 1 2 3 4 5 6 ·105 100 101 102 103 Project size (model elements) Generationtime(s) MT (run 1) MT (runs 2–5) HT (run 1) HT (runs 2–5) MT: Modelio M2T HT: Hawk M2T Cold runs: 10-100x faster on Hawk Warm runs: Hawk scales better A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 23 / 27
  33. 33. Intro Integration Evaluation Conclusions M2T with Hawk: tradeoff 1 2 3 4 5 6 ·105 0 5 10 15 20 1 Project size (model elements) MT/(HTrun1+indexing) How many cold Modelio runs before Hawk makes more sense? For the largest project, the answer is 1! A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 24 / 27
  34. 34. Intro Integration Evaluation Conclusions Outline 1 Intro 2 Integration 3 Evaluation 4 Conclusions A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 25 / 27
  35. 35. Intro Integration Evaluation Conclusions Conclusions and future work Obtained results SOFTEAM wanted querying for their model collab server Hawk is now powering it: Licensing issues guided many design decisions Available as dashboards and advanced manual querying Hawk+EGL faster than Modelio+Jython for M2T-as-service Future work SOFTEAM: integrate Hawk in more places and provide querying capabilities for less technical users Further performance improvements in Hawk: Horizontal scalability (more RAM had diminishing returns) On-demand derived features to reduce indexing time Optimize OrientDB backend (much work done for v1.1!) A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 26 / 27
  36. 36. End of the presentation Questions? @antoniogado A García-Domínguez et al. Integration of a Graph-Based Model Indexer in Commercial Modelling Tools 27 / 27

×