Presentation on the brand new EMF-IncQuery 0.7, for Itemis.

1. EMF-­‐INCQUERY:
Efficient
Model
Queries
for
EMF
Tools
Tamás
Szabó,
István
Ráth
Budapest
University
of
Technology
and
Economics IncQuery
Labs
Ltd.
Fault
Tolerant
Systems
Research
Group

2. Overview
§ IntroducAon
§ EMF-­‐INCQUERY
overview
§ Performance
benchmarking
§ Roadmap
and
ideas
for
the
future

3. A
key
problem
of
MDE
§ Scalability
vs.
modeling
tools
o Issues
encountered
by
several
industrial
partners
using
tools
based
on
the
Eclipse
Modeling
Framework
(EMF)
o Modeling
scenarios
can
get
really
complex
really
quickly
• Instance
models
of
size
1-­‐2M
and
beyond
• Performance
issues
with
model
transformaLons
and
code
generators
have
an
adverse
effect
on
everyday
development
tasks
§ Scalability? Recognized
e.g.
by
AUTOSAR
tool
vendors
hOp://wiki.eclipse.org/Auto_IWG_WP2
o Complex
(meta)models
o Large
instance
models
o Complex
query
and
manipulaLon
scenarios

4. Focus:
model
queries
§ Model
queries:
“a
piece
of
code
that
retrieves
a
given
set
of
the
model”
§ Queries
are
at
the
heart
of
MDE
o Every
model
access/read
is
a
(simple)
query
o More
complex:
Views,
content
providers
o Most
complex:
Model
transformaAons,
code
generators,
…
§ Query
performance
(=
the
speed
of
content
retrieval)
is
crucial
o First
vs.
consecuAve
vs.
throughput
o Query
result
vs.
query
contents
§ Core
problems
1. Slow
query
engines
2. Considerable
programming
effort
necessary
to
get
complex
queries
right

5. EMF-­‐INCQUERY

6. EMF-­‐INCQUERY
Overview
§ A
model
query
engine
for
batch
and
live
queries
Language ExecuAon Tools
• Expressive
• Incremental • On-­‐the-­‐fly
well-­‐
graph
paOern-­‐based
evaluaAon
based
on
the
formedness
validaAon
query
language Rete
algorithm • Incremental
model
• Focus
on
• Instantaneous
processing
reusability:
query
response
for
complex
(synchronizaAon,
...)
libraries,
recursive
queries
over
large
• View
maintenance
paOern
calls models • ...
§ Origins
o 10+
years
of
R&D
experience
in
model
queries
&
transformaAons

7. Benefits
of
EMF-­‐INCQUERY
§ Makes
on-­‐the-­‐fly
well-­‐formedness
validaAon,
view
maintenance,
…
feasible
over
really
large
instance
models
§ Simplifies
wriAng
really
complex
queries
o Graph
paOern
language
o Highly
reusable
à query
libraries
§ Easy-­‐to-­‐integrate
into
exisAng
apps
o Works
with
any
EMF
domain
metamodel
o Integrates
through
Eclipse
standards
à works
with
many
EMF-­‐based
apps
out-­‐of-­‐the-­‐box

8. Architecture
Input
=
Model
contents
EMF
instance
+
EMF
noAficaAons
Rete
network
EMF-­‐INCQUERY
Base
Input
Intermedia
Output Change
PaOern
matcher
Query
engine
Output
=
Query
results
+
Query
result
deltas
(con<nuously
updated
as
the
model
changes)

9. Architecture
Input
=
Model
contents
EMF
instance
+
EMF
noAficaAons
Model
ModificaAon
Rete
network
EMF-­‐INCQUERY
Base
Input
Intermedia
Output Change
PaOern
matcher
Query
engine
Output
=
Query
results
+
Query
result
deltas
(con<nuously
updated
as
the
model
changes)

10. IncQuery
Base
Indexers
§ Stand-­‐alone
core
library
with
minimal
dependencies
§ Provides
high
performance
indexers
o Typeàall
instances
o EdgeàSource,Target
o NodeàIncoming/Outgoing
edges
o AUribute
valueàhost
§ Supports
Incremental
TransiLve
Closure
o Efficiently
compute
&
maintain
reachability
regions
along
EReference
navigaLon
paths
§ Indexer
iniLalizaLon
o According
to
visitor
strategy
o Can
be
• Generic
(all
known
Ecore
types
are
indexed)
[“wildcard
mode”]
• Or
filtered
according
to
a
set
of
types
[memory
opLmizaLon
for
generated
query
code]

11. DEMO INCQUERY
PaOern
Language
§ School
metamodel
in
EMF

12. INCQUERY
PaOern
Language

13. INCQUERY
PaOern
Language
Expressive
declaraAve
query
language
by
graph
paOerns
Capture
local
+
global
queries
ComposiAonality
+
Reusabilility
Recursion,
NegaAon

14. Feature
highlights
§ State-­‐of-­‐the-­‐art
development
tools
based
on
Xtext2
o Content
assist
(code
compleAon),
validaAon,
outline,
…
o Advanced
(syntacAc
and
semanAc)
validaAon
§ Unique
query
language
features
o Recursive
queries
and
transiAve
closure
o Inference-­‐supported
type
system
and
expressions
based
on
Xbase
and
Ecore
o PaOern
composiAon
and
reuse
§ RunLme
o Build
and
execute
queries
on-­‐the-­‐fly
o Strong
generaAve
bindings
to
Java
§ Extension
APIs
o Query-­‐based
derived
features
with
noAficaAons
o ValidaAon
Engine
for
efficient
on-­‐the-­‐fly
validaAon
o Event-­‐driven
Virtual
Machine
for
transformaAons

15. INCQUERY
Development
Tools

16. INCQUERY
Development
Tools
PaOern
Editor
Query
Explorer

17. INCQUERY
Development
Tools
PaOern
Editor
Query
Explorer

18. INCQUERY
Development
Tools
• Works
with
most
EMF-­‐
based
editors
out-­‐of-­‐
the-­‐box
• Reveals
matches
as
selecAon
PaOern
Editor
Queries
are
applied
&
updates
on-­‐the-­‐fly
Query
Explorer

19. IntegraAon
opAons
§ IncQuery
Java
API
o Generic
(on-­‐the-­‐fly
queries)
o Generated
code
(opLmized
for
performance)
§ Query-­‐based
derived
features
o Edges
and
aUribute
values
computed
and
maintained
by
IncQuery
o Transparent
integraLon
through
genmodeled
EMF
o Unique
feature:
Efficient
and
automated
noLficaLon
support
§ Add-­‐on
APIs
o IncQuery
ValidaLon
(Demo)
o IncQuery
Databinding

20. IncQuery
Databinding
1.
Model
3.a
Result
ModificaAon 4.a
Add/Remove
items
Update
Query (list,
table,
tree…)
Match
Set
UI
Model Query Component
2.
Change
NoAficaAon
Match
3.b
Match 4.b
Update
Items
(Property) (list,
table,
tree
item,
Update
label,
…)
JFace
Databinding

21. Work
in
progress:
IncQuery
Viewers

22. Work
in
progress:
IncQuery
Viewers
1.
Define
“edges”
and
“nodes”
of
an
overlay
graph
as
queries

23. Work
in
progress:
IncQuery
Viewers
2.
Create
automaAcally
updaAng
JFace
viewers
(List,
Tree,
1.
Define
“edges”
Table,
Zest
Graph)
and
“nodes”
of
an
without
manual
overlay
graph
as
coding
queries

24. PERFORMANCE
BENCHMARK
Instantaneous
result
retrieval?

25. Benchmarking
scenario
§ In-­‐memory
models
o Embedded
railways
control
sodware
system
design
domain
(complex
queries)
o Instance
models
up
to
2.7M
EObjects
and
11.1M
EReferences
§ Scenario:
model
transformaLon
combined
with
well-­‐
formedness
validaLon
o Batch:
loading
models
+
execuLng
queries
o On-­‐the-­‐fly:
transforming
models
+
re-­‐execuLng
queries
instantaneously
§ Tools
o Eclipse
tools
(OCL,
OCL
Impact
Analyzer,
INCQUERY,
EMF-­‐Java
manually
opLmized)
o Neo4j
in-­‐memory
graph
database
(for
comparison)

26. Batch
operaAons

27. Batch
operaAons
Neo4j
scales
somewhat
beOer
than
top
EMF
tools
…
unless
you
augment
EMF
with
INCQUERY

28. Batch
operaAons
Neo4j
scales
somewhat
beOer
than
top
EMF
tools
Note
the
logarithmic
scales:
7x
faster!
…
unless
you
augment
EMF
with
INCQUERY

29. Real
strength:
on-­‐the-­‐fly
model
processing

30. Real
strength:
on-­‐the-­‐fly
model
processing
• IncQuery
provides
near
zero
response
Ames
for
up
to
14M
elements

31. Real
strength:
on-­‐the-­‐fly
model
processing
• IncQuery
provides
near
zero
response
Ames
for
up
to
14M
elements
CharacterisAc
difference

32. Memory
usage
§ Depends
on
“query
complexity”
and
model
size/
contents
§ For
typical
query/model
combinaAons,
memory
overhead
is
linear
o (1.1-­‐2)
x
(heap
required
for
the
model
itself)
o IncQuery’s
built-­‐in
semanAc
validators
and
the
Rete
Visualizer
aid
you
in
resolving
problemaAc
query/
model
combinaAons

33. APPLICATIONS
AND
ROADMAP

34. Known
applicaAons
§ Well-­‐formedness
validaAon
of
very
large,
industrial
AUTOSAR
models
(AutomoAve
OEM)
§ Incremental
dependency
analysis
over
sotware
infrastructure
graphs
(tool
used
in
CERN)
§ Queries,
views
and
transformaAons
for
model-­‐
driven
tool
development
in
the
avionics
domain
(aircrat
manufacturer)
§ Model-­‐driven
refactoring
of
large
source
code
models
(CollaboraAve
naAonal
project
in
Hungary)

35. Development
agenda
§ Our
current
plan
o IncQuery
Viewers
o Dynamic
EMF
support
o Xcore
integraAon
(query-­‐based
derived
features)
o Query
performance
opAmizaAon
o Improve
support
for
stereotyped
UML
models
§ ARTop/Sphinx
integraAon
o Incremental
validaAon
using
“hybrid
queries”
• Support
parAally
materialized
models

36. Other
ideas
on
the
table
§ ParLal
support
for
OCL
§ Dedicated
support
for
Complex
Event
Processing
over
live
EMF
models
o hUp://incquery.net/incquery/demos/jnect
o Think
incremental
queries
over
the
E4
Modeled
UI…
§ Develop
a
high
performance
and
pragmaLc
model
transformaLon
engine
o Build
the
language
on
Xbase
and
IQPL
o …and
the
runLme
on
IncQuery
§ Design
space
exploraLon
o OpLmize
instance
models
towards
a
complex
design
goal

37. Final
points
§ EMF-­‐INCQUERY
has
been
successfully
migrated
to
Eclipse.org
hUp://eclipse.org/incquery
§ EMF-­‐INCQUERY
0.7
milestone
builds
are
available
immediately
from
the
Marketplace
o Blog
and
other
resources:
hOp://incquery.net
o Lots
of
documentaAon:
examples,
demos,
tutorials
§ Release
0.7
is
tentaAvely
scheduled
shortly
ater
the
Eclipse
Kepler
Launch
(beginning
of
July)
§ Your
contribu<ons
(feedback,
forum
posts,
ideas,
bugzillas,
patches)
are
very
welcome!

38. Acknowledgements
§ IncQuery
Development
Team
o Gábor
Bergmann,
Ábel
Hegedüs,
Ákos
Horváth,
András
Ökrös,
Zoltán
Ujhelyi,
István
Ráth
§ AddiLonal
contributors
o Márk
CzoUer,
Tamás
Szabó
§ Performance
benchmarking
o Benedek
Izsó,
Zoltán
Szatmári
§ TesLng,
examples
o Asla
Csicsely,
Tamás
Csurgó,
Dániel
Kávássy,
Gábor
Szárnyas,
Tamás
Tóth,
Ádám
Dudás,
Oszkár
Semeráth,
Zoltán
Balogh
§ IncQuery
Project
Lead:
István
Ráth
§ Senior
advisor:
Dániel
Varró