Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lessons learned from building Eclipse-based add-ons for commercial modeling tools

17 views

Published on

In this presentation, we summarize the lessons we have learned during the MagicDraw adaptation of VIATRA, Eclipse’s open source framework for scalable reactive model transformations. We have built V4MD, an open source extension for MagicDraw that others can freely reuse and build on, and IncQuery for MagicDraw, a commercial add-on that provides powerful yet user-friendly querying and validation capabilities.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Lessons learned from building Eclipse-based add-ons for commercial modeling tools

  1. 1. EclipseCon France – June 14 2018 Lessons learned from building Eclipse-based add-ons for commercial modeling tools (from a technology perspective) István Ráth Ákos Horváth
  2. 2. MagicDraw • A popular modeling tool for UML/SysML, available since 1998 • Over 500.000 downloads in 90 countries • Standard-compliant and highly customizable platform • Not just a desktop app, but a complete suite of tools • Simulation • Analysis • Collaboration ! Teamwork Cloud
  3. 3. OpenMBEE: an open source ecosystem • Open modeling platform • Developed by NASA JPL and others • Provides • MMS – collaborative repository with OpenAPI/ Swagger interfaces • MDK – MagicDraw client built with MD OpenAPI • DocGen – document generator for SysML models • View Editor – web-based front-end • Integrations 
 (Mathematica, bae, K, DOORS-NG, Matlab) • A lot of useful and reusable code:
 https://github.com/Open-MBEE
  4. 4. IncQuery for MagicDrawTM and Teamwork CloudTM
  5. 5. IncQuery – cloud-based modeling beyond EMF
  6. 6. IncQuery – cloud-based modeling beyond EMF Scalable Language tailored to models Validation and analytics features Hybrid database technolog y
  7. 7. IncQuery – cloud-based modeling beyond EMF Scalable Language tailored to models Validation and analytics features Hybrid database technolog y Tool / Repository Persistent index VIATRA query & xform 
 engine In-memory index
  8. 8. IncQuery – cloud-based modeling beyond EMF Scalable Language tailored to models Validation and analytics features Hybrid database technolog y Tool / Repository Persistent index VIATRA query & xform 
 engine In-memory index • MagicDraw & Teamwork Cloud • Traditional DBs • NoSQL & graph DB • GitHub, …
  9. 9. IncQuery – cloud-based modeling beyond EMF Scalable Language tailored to models Validation and analytics features Hybrid database technolog y Tool / Repository Persistent index VIATRA query & xform 
 engine In-memory index • MagicDraw & Teamwork Cloud • Traditional DBs • NoSQL & graph DB • GitHub, … • OpenAPI / Swagger interfaces • High-performance indexing • Change updates via callbacks
  10. 10. IncQuery – cloud-based modeling beyond EMF Scalable Language tailored to models Validation and analytics features Hybrid database technolog y Tool / Repository Persistent index VIATRA query & xform 
 engine In-memory index • MagicDraw & Teamwork Cloud • Traditional DBs • NoSQL & graph DB • GitHub, … • Clusterized NoSQL • On-demand materialization • Ecore only used for schema / ontology, no EMF instantiation • OpenAPI / Swagger interfaces • High-performance indexing • Change updates via callbacks
  11. 11. IncQuery – cloud-based modeling beyond EMF Scalable Language tailored to models Validation and analytics features Hybrid database technolog y Tool / Repository Persistent index VIATRA query & xform 
 engine In-memory index • MagicDraw & Teamwork Cloud • Traditional DBs • NoSQL & graph DB • GitHub, … 1000x faster than conventional DB technology • Clusterized NoSQL • On-demand materialization • Ecore only used for schema / ontology, no EMF instantiation • OpenAPI / Swagger interfaces • High-performance indexing • Change updates via callbacks
  12. 12. IncQuery – cloud-based modeling beyond EMF Scalable Language tailored to models Validation and analytics features Hybrid database technolog y Tool / Repository Persistent index VIATRA query & xform 
 engine In-memory index Containerized microservices ! Horizontal scaling in the cloud • MagicDraw & Teamwork Cloud • Traditional DBs • NoSQL & graph DB • GitHub, … 1000x faster than conventional DB technology • Clusterized NoSQL • On-demand materialization • Ecore only used for schema / ontology, no EMF instantiation • OpenAPI / Swagger interfaces • High-performance indexing • Change updates via callbacks
  13. 13. Scenarios • Model analysis and validation reporting 
 (~ “LGTM / FindBugs for models”) • E.g. check if the signals sent/received over a port correspond to the flow properties • identify cyclic activity calls • find deadlock states, i.e. incoming transitions but not outgoing • Etc. • Change impact analysis • If I change an upstream project, how will it affect downstream projects? • Fast queries ! realizable in pre-commit hooks • Ad-hoc model queries • Model comprehension • Reviews • Traceability analysis
  14. 14. IncQuery for MagicDraw and Teamwork Cloud
 Screenshots
  15. 15. IncQuery for MagicDraw and Teamwork Cloud
 Screenshots
  16. 16. IncQuery for MagicDraw and Teamwork Cloud
 Screenshots
  17. 17. IncQuery is built on open source http://eclipse.org/viatra Model query and transformation framework • Declarative • Scalable • Reactive Easy integration • Java & other JVM languages • Enabling libraries for
 open & commercial
 tools Major industrial users & partners:
  18. 18. IncQuery is built on open source http://eclipse.org/viatra Model query and transformation framework • Declarative • Scalable • Reactive Easy integration • Java & other JVM languages • Enabling libraries for
 open & commercial
 tools Major industrial users & partners: What’s new in VIATRA 2.0? • Simplification • Fewer third party dependencies • New language features • Lots of performance and memory optimizations • Java compatibility (Java 8 required & used on API, Java 9/10 supported) • Available in Eclipse Photon!
  19. 19. MagicDraw and EMF • Lesson #1 Custom EMF metamodel for UML • Incompatible with MDT.UML • Most visible difference: profile support • Applied profiles and stereotypes can be referenced by name • „string typing” • Lesson #2 Custom EMF implementation • Mostly EMF-compatible, including change notifications! (support incremental processing out-of-the box) • Custom memory management (loading-unloading in the background) • Custom transaction handling
  20. 20. MagicDraw and EMF • Lesson #1 Custom EMF metamodel for UML • Incompatible with MDT.UML • Most visible difference: profile support • Applied profiles and stereotypes can be referenced by name • „string typing” • Lesson #2 Custom EMF implementation • Mostly EMF-compatible, including change notifications! (support incremental processing out-of-the box) • Custom memory management (loading-unloading in the background) • Custom transaction handling V4MD: VIATRA for MagicDraw • Open source ”glue code” (EPL v2) • Demonstrates how to bind EMF-based tech to MagicDraw • Deployable as a MD plug-in built with Gradle • http://github.com/viatra/v4md
  21. 21. MagicDraw vs. OSGi • MD 18.x+ is based on Eclipse Equinox • not entirely: the application itself is, but… • Lesson #3 • MagicDraw plug-ins are not OSGi bundles • No classpath separation • MagicDraw API cannot be referenced as OSGi dependencies • Lesson #4 • Custom plugin.xml format • Incompatible with PDE
  22. 22. Lesson #5: Integrating Xtext into MagicDraw • Parsing infrastructure is easy to reuse, but... • As standalone Java application • Maven dependencies useful • ... a full-blown editor integration is tricky • Many dependencies, including SWT and JDT • Integrating SWT-based UI into Swing is not practical • Alternatives we considered: • Future proof: Web-based editor with LSP • Quick-and-dirty: Separate Eclipse RCP based application
  23. 23. Alternatives: LSP vs. RCP LSP • Deploy language server within MD ! dependency issues • Potential collusions with built-in jars • No mature LSP client for Swing UI • Monaco editor has issues with built- in browser
  24. 24. Alternatives: LSP vs. RCP LSP • Deploy language server within MD ! dependency issues • Potential collusions with built-in jars • No mature LSP client for Swing UI • Monaco editor has issues with built- in browser RCP-based „companion app”
  25. 25. Alternatives: LSP vs. RCP LSP • Deploy language server within MD ! dependency issues • Potential collusions with built-in jars • No mature LSP client for Swing UI • Monaco editor has issues with built- in browser RCP-based „companion app” • No dependency issues inside MagicDraw • But requires a 200MB RCP app (per platform)
  26. 26. Alternatives: LSP vs. RCP LSP • Deploy language server within MD ! dependency issues • Potential collusions with built-in jars • No mature LSP client for Swing UI • Monaco editor has issues with built- in browser RCP-based „companion app” • No dependency issues inside MagicDraw • But requires a 200MB RCP app (per platform) • Just works • Custom “protocol” for communication
  27. 27. Alternatives: LSP vs. RCP LSP • Deploy language server within MD ! dependency issues • Potential collusions with built-in jars • No mature LSP client for Swing UI • Monaco editor has issues with built- in browser RCP-based „companion app” • No dependency issues inside MagicDraw • But requires a 200MB RCP app (per platform) • Just works • Custom “protocol” for communication • Can host the language server • A path forward as technology matures
  28. 28. Alternatives: LSP vs. RCP LSP • Deploy language server within MD ! dependency issues • Potential collusions with built-in jars • No mature LSP client for Swing UI • Monaco editor has issues with built- in browser RCP-based „companion app” • No dependency issues inside MagicDraw • But requires a 200MB RCP app (per platform) • Just works • Custom “protocol” for communication • Can host the language server • A path forward as technology matures • Platform specific issues with window management
  29. 29. Alternatives: LSP vs. RCP LSP • Deploy language server within MD ! dependency issues • Potential collusions with built-in jars • No mature LSP client for Swing UI • Monaco editor has issues with built- in browser RCP-based „companion app” • No dependency issues inside MagicDraw • But requires a 200MB RCP app (per platform) • Just works • Custom “protocol” for communication • Can host the language server • A path forward as technology matures • Platform specific issues with window management
  30. 30. Simple UML path expressions UML Types Keywords in purple Check expressions A "companion app” for MagicDraw:
 The VIATRA Query Language Editor https://www.eclipse.org/viatra/documentation/tutorial.html 
 https://www.eclipse.org/viatra/documentation/query-language.html Pattern that lists SysML Blocks
  31. 31. Lesson #6: Building MagicDraw plugins Challenges • Collect jars from a MagicDraw installation • Non-standard solution • Execute new MagicDraw instance with plug-in • Starting an OSGi container is non-trivial • Add third-party dependencies • Maven dependencies • Plugin descriptors have to include all jar files by name • Separate installation descriptor as well in a different format • Handling generated code • VIATRA VQL language • Xtend language
  32. 32. Lesson #6: Building MagicDraw plugins Challenges • Collect jars from a MagicDraw installation • Non-standard solution • Execute new MagicDraw instance with plug-in • Starting an OSGi container is non-trivial • Add third-party dependencies • Maven dependencies • Plugin descriptors have to include all jar files by name • Separate installation descriptor as well in a different format • Handling generated code • VIATRA VQL language • Xtend language Solution: Gradle
  33. 33. Lesson #6: Building MagicDraw plugins Challenges • Collect jars from a MagicDraw installation • Non-standard solution • Execute new MagicDraw instance with plug-in • Starting an OSGi container is non-trivial • Add third-party dependencies • Maven dependencies • Plugin descriptors have to include all jar files by name • Separate installation descriptor as well in a different format • Handling generated code • VIATRA VQL language • Xtend language Solution: Gradle • Originates from OpenMBEE project • Gradle-based solution • Relies on scripting
  34. 34. Lesson #6: Building MagicDraw plugins Challenges • Collect jars from a MagicDraw installation • Non-standard solution • Execute new MagicDraw instance with plug-in • Starting an OSGi container is non-trivial • Add third-party dependencies • Maven dependencies • Plugin descriptors have to include all jar files by name • Separate installation descriptor as well in a different format • Handling generated code • VIATRA VQL language • Xtend language Solution: Gradle • Originates from OpenMBEE project • Gradle-based solution • Relies on scripting • MagicDraw OpenAPI dependencies • Download and install MagicDraw instance • Use as dependency source
  35. 35. Lesson #6: Building MagicDraw plugins Challenges • Collect jars from a MagicDraw installation • Non-standard solution • Execute new MagicDraw instance with plug-in • Starting an OSGi container is non-trivial • Add third-party dependencies • Maven dependencies • Plugin descriptors have to include all jar files by name • Separate installation descriptor as well in a different format • Handling generated code • VIATRA VQL language • Xtend language Solution: Gradle • Originates from OpenMBEE project • Gradle-based solution • Relies on scripting • MagicDraw OpenAPI dependencies • Download and install MagicDraw instance • Use as dependency source • External dependencies • Downloaded via standard Gradle mechanisms • Script can update plugin and installation descriptor files
  36. 36. Lesson #6: Building MagicDraw plugins Challenges • Collect jars from a MagicDraw installation • Non-standard solution • Execute new MagicDraw instance with plug-in • Starting an OSGi container is non-trivial • Add third-party dependencies • Maven dependencies • Plugin descriptors have to include all jar files by name • Separate installation descriptor as well in a different format • Handling generated code • VIATRA VQL language • Xtend language Solution: Gradle • Originates from OpenMBEE project • Gradle-based solution • Relies on scripting • MagicDraw OpenAPI dependencies • Download and install MagicDraw instance • Use as dependency source • External dependencies • Downloaded via standard Gradle mechanisms • Script can update plugin and installation descriptor files • Generated code • Handled via Gradle plugins
  37. 37. Lesson #6: Building MagicDraw plugins Challenges • Collect jars from a MagicDraw installation • Non-standard solution • Execute new MagicDraw instance with plug-in • Starting an OSGi container is non-trivial • Add third-party dependencies • Maven dependencies • Plugin descriptors have to include all jar files by name • Separate installation descriptor as well in a different format • Handling generated code • VIATRA VQL language • Xtend language Solution: Gradle • Originates from OpenMBEE project • Gradle-based solution • Relies on scripting • MagicDraw OpenAPI dependencies • Download and install MagicDraw instance • Use as dependency source • External dependencies • Downloaded via standard Gradle mechanisms • Script can update plugin and installation descriptor files • Generated code • Handled via Gradle plugins • Running MagicDraw • Starts the Platform Runner of MagicDraw • Parameterized with the Gradle Java Runner
  38. 38. Lesson #6: Building MagicDraw plugins Challenges • Collect jars from a MagicDraw installation • Non-standard solution • Execute new MagicDraw instance with plug-in • Starting an OSGi container is non-trivial • Add third-party dependencies • Maven dependencies • Plugin descriptors have to include all jar files by name • Separate installation descriptor as well in a different format • Handling generated code • VIATRA VQL language • Xtend language Solution: Gradle • Originates from OpenMBEE project • Gradle-based solution • Relies on scripting • MagicDraw OpenAPI dependencies • Download and install MagicDraw instance • Use as dependency source • External dependencies • Downloaded via standard Gradle mechanisms • Script can update plugin and installation descriptor files • Generated code • Handled via Gradle plugins • Running MagicDraw • Starts the Platform Runner of MagicDraw • Parameterized with the Gradle Java Runner MD Plugin Skeleton • “Hello world” plugin based on the Gradle setup • Easy to reuse • https://github.com/IncQueryLabs/MD_plugin_skeleton
  39. 39. Some more open source contributions • MD VIATRA benchmark • https://github.com/IncQueryLabs/magicdraw-viatra-benchmark • Scalability benchmark for VIATRA queries over MagicDraw models • TMT model fork • https://github.com/IncQueryLabs/TMT-SysML-Model • Large (300k+) real-world SysML model, very useful to testing tools • Examples of actual IQ4MD validation rules inspired by NASA JPL • MDK fork • https://github.com/IncQueryLabs/mdk • Example usage of V4MD within MDK, NASA JPL’s toolkit
  40. 40. Takeaways
  41. 41. Takeaways • Integrating with proprietary tools is more challenging... • Compared to a purely open Eclipse ecosystem • ... But not that hard with the proper tools
  42. 42. Takeaways • Integrating with proprietary tools is more challenging... • Compared to a purely open Eclipse ecosystem • ... But not that hard with the proper tools • No Magic is an open and collaborative partner • Open ecosystem around MD (on a smaller scale) is worth exploring • And contributing to!
  43. 43. Takeaways • Integrating with proprietary tools is more challenging... • Compared to a purely open Eclipse ecosystem • ... But not that hard with the proper tools • No Magic is an open and collaborative partner • Open ecosystem around MD (on a smaller scale) is worth exploring • And contributing to! • Watch out for VIATRA 2.0 and the IncQuery Server
  44. 44. Takeaways • Integrating with proprietary tools is more challenging... • Compared to a purely open Eclipse ecosystem • ... But not that hard with the proper tools • No Magic is an open and collaborative partner • Open ecosystem around MD (on a smaller scale) is worth exploring • And contributing to! • Watch out for VIATRA 2.0 and the IncQuery Server • Get the IncQuery MagicDraw plug-in at https:// incquerylabs.com/incquery • Complete with tutorial and examples
  45. 45. Takeaways • Integrating with proprietary tools is more challenging... • Compared to a purely open Eclipse ecosystem • ... But not that hard with the proper tools • No Magic is an open and collaborative partner • Open ecosystem around MD (on a smaller scale) is worth exploring • And contributing to! • Watch out for VIATRA 2.0 and the IncQuery Server • Get the IncQuery MagicDraw plug-in at https:// incquerylabs.com/incquery • Complete with tutorial and examples
  46. 46. Thank you! +36 70 633 3973 @IncQueryLabs iq4md AT incquerylabs.com http://iq4md.incquerylabs.com info AT incquerylabs.com

×