This document discusses incremental and streaming model transformations. It describes how models are increasingly large and dynamic, necessitating incremental and streaming approaches to model transformations. Batch transformations are simple but inefficient, while incremental transformations are more complex but efficient by caching and updating matches. The document also describes how models may be streamed as sensor or generated data, requiring streaming transformations over live models. Techniques discussed include processing model fragments and using complex event processing to evaluate model changes.

1. Incremental and Streaming
Model Transformations
Istvan David

2. A case for incrementality
• MT rule: precondition  postcondition
Querying
Evolution of large (instance) models
Re-evaluation of MT preconditions as the model changes
Batch vs Incremental
Example: live model validation
Maintaining live models, models@run.time, managing streamed models,
live M2M synchronization...

3. A case for incrementality
• MT rule: precondition  postcondition
Querying
Evolution of large (instance) models
Re-evaluation of MT preconditions as the model changes
Batch vs Incremental
Example: live model validation
Maintaining live models, models@run.time, managing streamed models,
live M2M synchronization...
How big are models?
Markus Scheidgen: How Big Are Models – An Estimation (2012)
Application Model size
System models 108
Sensor data 109
Geospatial models 1012

4. Batch vs Incremental
• Efficiency vs complexity
• Batch: easy to implement
• Incremental: efficient execution, but very complex algorithms required
Typical in state-of-the-
practice tools!
...on the querying side

5. Batch vs Incremental
• Efficiency vs complexity
• Batch: easy to implement
• Incremental: efficient execution, but very complex algorithms required
• Incremental graph pattern matching
• More space, less time
• Cache matches of graph patterns
• Update cache upon changes
• Notify about relevant changes
• RETE, LEAPS, TREAT...
Typical in state-of-the-
practice tools!
...on the querying side
Source: D Varró, Incremental evaluation of model queries, Presentation @DSM-TP 2015,
http://msdl.cs.mcgill.ca/conferences/dsm-tp-
2015/material/15.08.26.IncrementalQueries.DanielVarro.pdf

6. Batch vs Incremental
• Efficiency vs complexity
• Batch: easy to implement
• Incremental: efficient execution, but very complex algorithms required
• Incremental graph pattern matching
• More space, less time
• Cache matches of graph patterns
• Update cache upon changes
• Notify about relevant changes
• RETE, LEAPS, TREAT...
Typical in state-of-the-
practice tools!
...on the querying side
Source: D Varró, Incremental evaluation of model queries, Presentation @DSM-TP 2015,
http://msdl.cs.mcgill.ca/conferences/dsm-tp-
2015/material/15.08.26.IncrementalQueries.DanielVarro.pdf
No
incrementality
Dirty
incrementality
Incrementality
by traceability
„Reactive” MTs
Large-step, re-execute from
scratch only for the changed
models
Small-step, detect missing
traceability links
Detect new activations, fire rule
activations

7. A case for incrementality
• MT rule: precondition  postcondition
Querying
Evolution of large (instance) models
Re-evaluation of MT preconditions as the model changes
Batch vs Incremental
Example: live model validation
Maintaining live models, models@run.time, managing streamed models,
live M2M synchronization...
How big are models?
Markus Scheidgen: How Big Are Models – An Estimation (2012)
Application Model size
System models 108
Sensor data 109
Geospatial models 1012
Assumption: models
are materialized
...what if they are
not?

8. Streaming MTs
• Models aren’t completely materialized
• Native (or natural) streams
• Sensor systems, CPS, cloud computing...
• Artificial (generated) streams
• Model split up and pushed through a channel
• Large models (AUTOSAR, etc)
• Streaming MTs over live models [1]
“a special kind of transformation in which the whole input model is not
completely available at the beginning of the transformation, but it is
continuously generated”
Sánchez Cuadrado, J., Lara, J.: Streaming model transformations:
Scenarios, challenges and initial solutions (2013) In Duddy, K., Kappel,
G., eds.: Theory and Practice of Model Transformations. Volume 7909
of Lecture Notes in Computer Science. Springer Berlin Heidelberg
(2013) 116

9. Streaming MTs
• Models aren’t completely materialized
• Native (or natural) streams
• Sensor systems, CPS, cloud computing...
• Artificial (generated) streams
• Model split up and pushed through a channel
• Large models (AUTOSAR, etc)
• Streaming MTs over live models [1]
[1] I. Dávid, I. Ráth, and D. Varró, Streaming model transformations by complex event processing, MoDELS 2014,
Conference acceptance rate: 26%. Invited for submission to MODELS14 Special Issue in SoSyM

10. Techniques
• Model fragment processing (Lara et al)
• Actual fragments of models are streamed
• Fragments are assembled on the target side
• Ongoing work, implementation planned via Apache Spark
• Related: P2P distributed models (Hartman et al [2])
• Complex event processing
• Model change information streamed as a series of events
• Evaluation of temporal and timing relationships
• Implemented within the VIATRA framework [3]
[2] Hartman et al, Stream my Models: Reactive Peer-to-Peer Distributed Models@run.time, MoDELS 2015
[3] http://www.eclipse.org/viatra/

11. Pointers
• Incremental MTs
• Tools: IncQuery, ATL, Epsilon...
• Background: RETE (Forgy, 1983), other „custom” cache algorithms
• A. Kusel et al, A Survey on Incremental Model Transformation Approaches,
Workshop on Models and Evolution 2013
• Streaming MTs
• Sánchez Cuadrado, J., Lara, J, Streaming model transformations: Scenarios,
challenges and initial solutions, 2013
• I. Dávid, I. Ráth, D. Varró, Streaming model transformations by complex event
processing, MoDELS 2014
• https://wiki.eclipse.org/VIATRA/CEP
• Hartman et al, Stream my Models: Reactive Peer-to-Peer Distributed
Models@run.time, MoDELS 2015