IncQuery Server
for Teamwork Cloud
Scalable Query Evaluation over
Collaborative Model Repositories
István Ráth
CEO
About us
• IncQuery Labs
• Specialist company focused on high-tech so:ware innova=on
around modeling and MBSE
• Based in Budapest, Hungary
• Founded in 2012 as a university spinoff
• Solu=ons
• Custom development, toolchain integra=on, cer=fica=on, consul=ng
• Deep tech enablers in model storage / processing / versioning
• Commercial support for open source so:ware
(Eclipse Modeling, OpenMBEE, …)
Development partner
Our Partners
Most Influential
Technology Partner Award
@No Magic World
Symposium 2018
Context: Large-scale and collaborative MBSE
Airframers Car Manufacturers
System Designers and Integrators
SW Supplier 1
SW Supplier 2
Certification Authorities Platform Provider
Smart & Safe
Cyber-Physical Systems
Characteristics:
• Different stakeholders
• Complex system models
• Collaborative processes
• IP protection
Needs for scalable
collaborative tools
• Large models
• Version control
• Access control
• Traceability
• ...
Challenges
• Project Manager : Is the model of my team consistent?
• Model Analysis, Validation and Error Reporting
• Project Engineer : Which other projects will be affected by my change and
how?
• Change Impact Analysis
• Reviewer : What is the content of your team’s model?
• Evaluation of Custom Queries
MagicDraw and Teamwork Cloud
• A popular modeling tool for UML/SysML, available since 1998
• Over 500.000 downloads in 90 countries
• Standard-compliant and highly customizable platform
• Not just a desktop app, but a complete suite of tools
• Simulation
• Analysis
• Collaboration à Teamwork Cloud
Teamwork Cloud
Repository
MagicDraw
(desktop
authoring)
Cameo
Collaborator
(web
collabora=on)
IncQuery: a query engine built for large-scale modeling
Scalable
Language
tailored to
models
Validation
and
analytics
features
Hybrid
database
technology
• Persistent and
in-memory
• Distributed
architecture
• 10-100x faster than
conventional DB
technology
• Elastic
(Docker/Kubernetes)
• VQL: Similar to SPARQL,
but tailored for SysML
• Adaptable to other
modeling languages
(SysML profiles,
AUTOSAR, Capella, etc)
Teamwork Cloud
Repository
MagicDraw
(desktop
authoring)
IncQuery Server
Cameo
Collaborator
(web
collaboraSon)
• Built-in and
custom queries
• Executable on
revisions and
enSre
repositories
Indexing
Querying
• Auxiliary back-end service
• Provides value added funcSonality
to MD and CC clients
• Fully integrated access control
Performance
• Indexing
• 1.2M model: <20s thanks to delta processing
• 30M repository: <1 hour thanks to delta processing
Performance
• Indexing
• 1.2M model: <20s thanks to delta processing
• 30M repository: <1 hour thanks to delta processing
• Querying
• Complete validation suite: 1.2M model: <1min
• Impact analysis: 30M repository: < 10s
• Custom queries: <0.5s for most queries, even for large models
• At least 10x faster than state-of-the-art graph / linked data databases
(Neo4j, Amazon Neptune, …)
Web-based APIs and User Interfaces
OpenAPI
interfaces
Easy-to-use developer console
• Server-side validation
• Results as JSON or HTML
• Subject to scoping and
access control
IncQuery for MagicDraw
Query authoring add-on
• Complete with tutorial and examples
• Powerful model queries
• Comprehensive active validation
support
• Build your own active validation rule
suites
• Use your own opaque behaviors for
quick fixes
• Fully integrated with MagicDraw 18.5
and 19.x
IncQuery Server
SPARQL support via Amazon Neptune
IncQuery as a tool integration platform
TWC
Discover
Model
crawler
Index
Persistent
index
In-memory
index
Query
Query
engine
Transform
RDF
exporter
Neptune
S3
Development Roadmap
• IncQuery 7 – Q4 2018
• Graph queries
• Query-based Validation, Smart Packages
and Relation Maps
• Impact Analysis
• IncQuery 8 – Q2 2019
• RDF / SPARQL support
• Full-text search
• Jupyter front-end integration
• IncQuery 9 – Q4 2019
• Open API for additional data sources
• Open API for custom transformations
https://incquerylabs.com/incquery
https://incquerylabs.com/in
IncQuery Labs ❤ open source
Model query and transformation framework
• Declarative
• Scalable
• Reactive
Easy integration
• Java & other JVM languages
• Enabling libraries for
open & commercial
tools
Major industrial users & partners:
http://eclipse.org/viatra
Open source projects by IncQuery Labs
in the OpenMBEE ecosystem
Features
• V4MD
• http://github.com/viatra/v4md
• MagicDraw bindings for Eclipse VIATRA
• MD_plugin_skeleton
• https://github.com/IncQueryLabs/MD_plugin_sk
eleton
• Basic MagicDraw plug-in built using Gradle
(inspired by MDK)
• Demonstrates the usage of V4MD
• MDK fork
• https://github.com/IncQueryLabs/mdk
• Example usage of V4MD within MDK
• MMS support for IncQuery
• Neptune (SPARQL) / Jupyter integration
• 2019 Q3
Benchmarks and demos
• MD VIATRA benchmark
• https://github.com/IncQueryLabs/magicdraw-
viatra-benchmark
• Scalability benchmark for VIATRA queries over
MagicDraw models
• Based on scaled-up variants of TMT
• TMT model fork
• https://github.com/IncQueryLabs/TMT-SysML-
Model
• Examples of custom complex validation queries
inspired by NASA JPL
Thank you!
@IncQueryLabs
incquery@incquerylabs.com
https://incquerylabs.com/incquery
info@incquerylabs.com

IncQuery Server for Teamwork Cloud - Talk at IW2019

  • 1.
    IncQuery Server for TeamworkCloud Scalable Query Evaluation over Collaborative Model Repositories István Ráth CEO
  • 2.
    About us • IncQueryLabs • Specialist company focused on high-tech so:ware innova=on around modeling and MBSE • Based in Budapest, Hungary • Founded in 2012 as a university spinoff • Solu=ons • Custom development, toolchain integra=on, cer=fica=on, consul=ng • Deep tech enablers in model storage / processing / versioning • Commercial support for open source so:ware (Eclipse Modeling, OpenMBEE, …)
  • 3.
    Development partner Our Partners MostInfluential Technology Partner Award @No Magic World Symposium 2018
  • 4.
    Context: Large-scale andcollaborative MBSE Airframers Car Manufacturers System Designers and Integrators SW Supplier 1 SW Supplier 2 Certification Authorities Platform Provider Smart & Safe Cyber-Physical Systems Characteristics: • Different stakeholders • Complex system models • Collaborative processes • IP protection Needs for scalable collaborative tools • Large models • Version control • Access control • Traceability • ...
  • 5.
    Challenges • Project Manager: Is the model of my team consistent? • Model Analysis, Validation and Error Reporting • Project Engineer : Which other projects will be affected by my change and how? • Change Impact Analysis • Reviewer : What is the content of your team’s model? • Evaluation of Custom Queries
  • 6.
    MagicDraw and TeamworkCloud • A popular modeling tool for UML/SysML, available since 1998 • Over 500.000 downloads in 90 countries • Standard-compliant and highly customizable platform • Not just a desktop app, but a complete suite of tools • Simulation • Analysis • Collaboration à Teamwork Cloud Teamwork Cloud Repository MagicDraw (desktop authoring) Cameo Collaborator (web collabora=on)
  • 7.
    IncQuery: a queryengine built for large-scale modeling Scalable Language tailored to models Validation and analytics features Hybrid database technology • Persistent and in-memory • Distributed architecture • 10-100x faster than conventional DB technology • Elastic (Docker/Kubernetes) • VQL: Similar to SPARQL, but tailored for SysML • Adaptable to other modeling languages (SysML profiles, AUTOSAR, Capella, etc) Teamwork Cloud Repository MagicDraw (desktop authoring) IncQuery Server Cameo Collaborator (web collaboraSon) • Built-in and custom queries • Executable on revisions and enSre repositories Indexing Querying • Auxiliary back-end service • Provides value added funcSonality to MD and CC clients • Fully integrated access control
  • 8.
    Performance • Indexing • 1.2Mmodel: <20s thanks to delta processing • 30M repository: <1 hour thanks to delta processing
  • 9.
    Performance • Indexing • 1.2Mmodel: <20s thanks to delta processing • 30M repository: <1 hour thanks to delta processing • Querying • Complete validation suite: 1.2M model: <1min • Impact analysis: 30M repository: < 10s • Custom queries: <0.5s for most queries, even for large models • At least 10x faster than state-of-the-art graph / linked data databases (Neo4j, Amazon Neptune, …)
  • 10.
    Web-based APIs andUser Interfaces OpenAPI interfaces Easy-to-use developer console • Server-side validation • Results as JSON or HTML • Subject to scoping and access control
  • 11.
    IncQuery for MagicDraw Queryauthoring add-on • Complete with tutorial and examples • Powerful model queries • Comprehensive active validation support • Build your own active validation rule suites • Use your own opaque behaviors for quick fixes • Fully integrated with MagicDraw 18.5 and 19.x
  • 12.
    IncQuery Server SPARQL supportvia Amazon Neptune IncQuery as a tool integration platform TWC Discover Model crawler Index Persistent index In-memory index Query Query engine Transform RDF exporter Neptune S3
  • 13.
    Development Roadmap • IncQuery7 – Q4 2018 • Graph queries • Query-based Validation, Smart Packages and Relation Maps • Impact Analysis • IncQuery 8 – Q2 2019 • RDF / SPARQL support • Full-text search • Jupyter front-end integration • IncQuery 9 – Q4 2019 • Open API for additional data sources • Open API for custom transformations https://incquerylabs.com/incquery https://incquerylabs.com/in
  • 14.
    IncQuery Labs ❤open source Model query and transformation framework • Declarative • Scalable • Reactive Easy integration • Java & other JVM languages • Enabling libraries for open & commercial tools Major industrial users & partners: http://eclipse.org/viatra
  • 15.
    Open source projectsby IncQuery Labs in the OpenMBEE ecosystem Features • V4MD • http://github.com/viatra/v4md • MagicDraw bindings for Eclipse VIATRA • MD_plugin_skeleton • https://github.com/IncQueryLabs/MD_plugin_sk eleton • Basic MagicDraw plug-in built using Gradle (inspired by MDK) • Demonstrates the usage of V4MD • MDK fork • https://github.com/IncQueryLabs/mdk • Example usage of V4MD within MDK • MMS support for IncQuery • Neptune (SPARQL) / Jupyter integration • 2019 Q3 Benchmarks and demos • MD VIATRA benchmark • https://github.com/IncQueryLabs/magicdraw- viatra-benchmark • Scalability benchmark for VIATRA queries over MagicDraw models • Based on scaled-up variants of TMT • TMT model fork • https://github.com/IncQueryLabs/TMT-SysML- Model • Examples of custom complex validation queries inspired by NASA JPL
  • 16.