Incremental pattern matching in the VIATRA2 model transformation system
Incremental pattern matching in the VIATRA d lt f ti tVIATRA model transformation systemGábor BergmannAndrás ÖkrösIstván Ráth (email@example.com)Dániel VarróDániel Varró Department of Measurement and Department of Measurement andGergely Varró Information Systems Budapest University of Technology and University of Technology and Economics
Overview• Introduction• Concepts•P f Performance analysis l i• Future work• Summary
Incremental model transformations model transformations• Key usage scenarios for MT: ▫ Mapping between languages ▫ Intra‐domain model manipulation Model execution Validity checking (constraint evaluation) Validity checking (constraint evaluation)• They work with evolving models. ▫ Users are constantly changing/modifying them. ▫ Users usually work with large models. Users usually work with large• Problem: transformations are slow ▫ To execute… (large models) ▫ and to re‐execute again and again (always starting from scratch).• Solution: incrementality ▫ Take the source model, and its mapped counterpart; ▫ Use the information about how the source model was changed; ▫ M Map and apply the changes (but ONLY the changes) to the target model. d l h h (b ONLY h h ) h d l
Towards incrementalityTowards incrementality• How to achieve incrementality? How to achieve incrementality? ▫ Incremental updates: avoid re‐generation. Don t recreate what is already there. Don’t recreate what is already there Use reference (correspondence) models. ▫ Incremental execution: avoid re‐computation. Incremental execution: avoid re computation. Don’t recalculate what was already computed. How?
Incremental graph pattern matchingIncremental graph pattern matching• Graph transformations require pattern matching p q p g• Goal: retrieve the matching set quickly• How? ▫ Store (cache) matchings ▫ Update them as the model changes Update precisely (incrementality) U d t i l (i t lit )• Expected results: good, if… ▫ There is enough memory ( ) There is enough memory (*) ▫ Queries are dominant ▫ Model changes are relatively sparse (**) ▫ e.g. synchronization, constraint evaluation, …
Operational overview XForm pattern interpreter model manipulation matching Incremental event VIATRA pattern notification Model space matcher updates
Architecture LS pattern LS tt Model XForm matcher parser parser XML XFormVIATRAV serializer interpreter Incremental Native importer & pattern A2 Framewo loader interface l d i t f matcher Core interfaces ork VIATRA Model space Program model store
Core idea: use RETE nets• RETE network INPUT ▫ node: (partial) matches of a node: (partial) matches of a (sub)pattern t3 Model space Model space p1 p2 p3 t1 t2 k1 k2 t3 ▫ edge: update propagation t3 t3 t3• Demonstrating the principle Demonstrating the principle ▫ input: Petri net Input nodes : Place p1 p2 p3 d : Token k1 k2 : Transition t1 t2 t3 ▫ pattern: fireable transition ▫ Model change: new transition t3 (t3) t1 Intermediate p1, k1 p2, k2 p1 nodes p1, k1, t1 p2, k2, t3 p3 3 p2 t2 p2, k2, t3 t3 Production node Production node p1, k1, t1, p3 p2, k2, t2, p3
RETE network constructionRETE network construction• Key: pattern decomposition y p p ▫ Pattern = set of constraints (defined over pattern variables) ▫ Types of constraints: type, topology (source/target), hierarchy (containment), attribute value, generics h h ( ) b l (instanceOf/supertypeOf), injectivity, [negative] pattern calls, … calls, …• Construction algorithm (roughly) ▫ 1. Decompose the pattern into elementary constraints (*) ▫ 2. Process the elementary constraints and connect them with appropriate intermediate nodes (JOIN, MINUS‐JOIN, UNION, …) UNION ) ▫ 3. Create terminator production node
Key RETE componentsKey RETE components INPUT• JOIN node JOIN node INPUT INPUT ▫ ~relational algebra: natural join natural join JOIN• MINUS‐JOIN ▫ Negative existence Negative existence JOIN PRODUCTION sourcePlace
Other VIATRA featuresOther VIATRA features• Pattern calls ▫ Simply connect the production nodes ▫ Pattern recursion is fully supported• OR patterns OR‐patterns ▫ UNION intermediate nodes• Check conditions ▫ check (value(X) % 5 == 3) ▫ check (length(name(X)) < 4) ▫ check (myFunction(name(X))!=‘myException’) check (myFunction(name(X))!= myException ) ▫ Filter and term evaluator nodes• Result: full VIATRA transformation language support; any pattern can be matched incrementally.
Updates• Needed when the model space changes Needed when the model space changes• VIATRA notification mechanism (EMF is also possible) ▫ Transparent: user modification, model imports, results of a Transparent: user modification, model imports, results of a transformation, external modification, … RETE is always updated!• Input nodes receive elementary modifications and release an update token ▫ Represents a change in the partial matching (+/‐)• Nodes process updates and propagate them if needed ▫ PRECISE update mechanism
Performance• In theory… ▫ Building phase is slow (“warm‐up”) How slow? ▫ Once the network is built, pattern matching is an ,p g “instantaneous” operation. Excluding the linear cost of reading the result set. ▫ But… there is a performance penalty on model manipulation. p p y p How much?• Dependencies? ▫ Pattern size Pattern size ▫ Matching set size ▫ Model size ▫ …? ?
Benchmarking• Example transformation: Petri net simulation ▫ One complex pattern for the enabledness condition ▫ Two graph transformation rules for firing ▫ As‐long‐as‐possible (ALAP) style execution (“fire at will”) ▫ Model graphs: A “large” Petri net actually used in a research project (~60 places, ~70 transitions, ~300 arcs) Scaling up: automatic generation preserving liveness ( t 100000 S li t ti ti i li (up to 100000 places, 100000 transitions, 500000 arcs)• Analysis ▫ Measure execution time (average multiple runs) Measure execution time (average multiple runs) ▫ Take “warm‐up” runs into consideration• Profiling ▫ Measure overhead network construction time Measure overhead, network construction time ▫ “Normalize” results
Profiling resultsProfiling results• Model manipulation overhead: ~15% (of overall CPU p ( time) ▫ Depends largely on the transformation!• Memory overhead Memory overhead ▫ Petri nets (with RETE networks) up to ~100000 fit into 1‐ 1.5GB RAM (VIATRA model space limitations) ▫ Grows linearly with model size (as expected) ( ) ▫ Nature of growth is pattern‐dependent• Network construction overhead Network construction overhead ▫ Similar to memory; pattern‐dependent. ▫ PN: In the same order as VIATRA’s LS heuristics initialization. initialization
Execution times Execution times Matches/outperforms Sparse Petri net benchmark GrGEN.NET for large models and high d l d hi h 1000000 iteration counts. Viatra/RETE Three orders of (x1k) ( 1k) 100000 magnitude and Viatra/LS (x1k) growing… ms) 10000 on time (m GrGenNET (x1k) G G NET ( 1k) 1000 Viatra/RETE Executio 100 (x1M) ( 1M) GrGen.NET 10 (x1M) 100 1000 10000 100000 Petri net size
Benchmarking summaryBenchmarking summary• Predictable near‐linear growth Predictable near linear growth ▫ As long as there is enough memory ▫ Certain problem classes: constant execution time ☺ Certain problem classes: constant execution time ☺ ▫ A ga
Improving performanceImproving performance• Strategies ▫ Improve the construction algorithm Memory efficiency (node sharing) Memory efficiency (node sharing) Heuristics‐driven constraint enumeration (based on p pattern [and model space] content) [ p ] ) ▫ Parallelism Update the RETE network in parallel with the transformation Parallel network construction ▫?
More benchmarking…More benchmarking• Ongoing research Ongoing research ▫ Extending the Varro benchmark Mutex STS/LTS ORM ▫ Extended benchmarking use cases Extended benchmarking use cases Simulation (model execution) Synchronization y Constraint evaluation ▫ Parallel transformations
Event‐driven live transformationsEvent‐driven live transformations• Problem: MT is mostly batch‐like y ▫ But models are constantly evolving Frequent re‐ transformations are needed for mapping synchronization constraint checking …• An incremental PM can solve the performance problem, but a formalism is needed ▫ to specify when to (re)act ▫ and how.• Ideally the formalism should be MT‐like Ideally, the formalism should be MT like.
Event‐driven live transformations (cont d)Event‐driven live transformations (cont’d)• An idea: represent events as model elements. p• Our take: represent events as changes in the matching set of a pattern. ▫ ~generalization• Live transformations ▫ maintain the context (variable values, global variables, …); i t i th t t ( i bl l l b l i bl ) ▫ run as a “daemon”, react whenever necessary; ▫ as the models change the system can react instantly since as the models change, the system can react instantly, since everything needed is there in the RETE network: no re‐ computation is necessary.• Paper accepted at ICMT2008.
Summary• Incremental pattern matching support integrated Incremental pattern matching support integrated into VIATRA2 R3 ▫ Based on the RETE algorithm Based on the RETE algorithm ▫ Provides full support for the pattern language ▫ High performance in certain problem classes High performance in certain problem classes• Future ▫ Performance will be further improved Performance will be further improved ▫ New applications in live transformations