Conference presentation for ICGT 2008.

1. Budapest University of Technology and Economics
A Benchmark Evaluation for
Incremental Pattern Matching in
Graph Transformation
Gábor Bergmann, Ákos Horváth,
István Ráth, Dániel Varró
Fault-tolerant Systems Research Group

2. Budapest University of Technology and Economics 2
Talk Overview
GT &
Introduction incremental
overview
Case studies Measurements
Fault-tolerant Systems Research Group ICGT '08

3. Budapest University of Technology and Economics 3
Benchmarking
 Aim:
− systematic and reproducible measurements
− on performance
− under different and precisely defined circumstances
 Overall goal:
− help transformation engineers in selecting tools
− Serve as reference for future research
 Popular approach in different fields
− AI
− relational databases
− rule-based expert systems
Fault-tolerant Systems Research Group ICGT '08

4. Budapest University of Technology and Economics 4
Benchmarking in graph transformation
 Specification examples for GT
− Goal: assessing expressiveness
− UML-to-XMI, object-relational mapping,
UML-to-EJB, etc.
 „Generic” Performance benchmarks for GT
− Varró benchmark
− R. Geiß and M. Kroll: On Improvements of the Varró
Benchmark for Graph Transformation Tools
− (Agtive Tool Contest, Grabats ’08, …)
 In our paper:
Benchmarks for graph transformation with
incremental pattern matching
− simulation
− synchronization
Fault-tolerant Systems Research Group ICGT '08

5. Budapest University of Technology and Economics 5
Talk overview
GT &
Introduction incremental
overview
Case Studies Measurements
Fault-tolerant Systems Research Group ICGT '08

6. Budapest University of Technology and Economics 6
Metamodeling
Class
1
At most one tokens Place
1
Association
* 1
outarc
Metamodel
Token Multiplicity
inarc
* * constraint
Arbitrary
Slot Transition
Instance model
t1:Transition Object
a4:outarc a3:inarc
p1:Place Link p3:Place
a1:inarc a2:outarc
tkn3:tokens
to1:Token t2:Transition
Fault-tolerant Systems Research Group ICGT '08

7. Budapest University of Technology and Economics 7
Graph Transformation
LHS RHS
a1:inarc a2:outarc a1:inarc a2:outarc
Place Tran. Place Place Tran. Plan
ttn1:tokens tkn2:tokens
Token Token
matching updating
Phases of GT matching
– Pattern Matching phase
– Updating phase: delete+ create
Pattern Matching is the most critical issue from
performance viewpoint
Fault-tolerant Systems Research Group

8. Budapest University of Technology and Economics 8
Incremental Pattern Matching
 Precondition of GT rules  pattern matching
 Goal
− Store matching sets
− Incremental update
− Fast response
 Good performance expected when:
− frequent pattern matching
− Small updates
 Possible application domain
− E.g. synchronization, constraints, model simulation, etc.
 An adapted RETE algorithm
Fault-tolerant Systems Research Group

9. Budapest University of Technology and Economics 9
Incremental Pattern Matching Example
• RETE net
– nodes: intermediate matchings INPUT
t3
– edge: update propagation p1p2p3 t1 t2 k1 k2 t3
• Example t3 t3 t3
– input: Petri net
Place Token Transition
– pattern: firable transition p1p2 p3 k1 k2 t1 t2 t3
– update: new transition
t3
p1, k1 p2, k2
t1
p1
p1, k1, t1 p2, k2, t3
p3 p2
t2 p2, k2, t3
p1, k1, t1, p3 p2, k2, t2, p3
t3
Fault-tolerant Systems Research Group

10. Budapest University of Technology and Economics 10
Talk overview
GT &
Introduction incremental
overview
Case Studies Measurements
Fault-tolerant Systems Research Group ICGT '08

11. Budapest University of Technology and Economics 11
Test Cases
 Model Simulation  Synchronization
− Visual languages − Match reusability
− Static graph structure − Unidirectional
− Small local model − Complex rules
manipulation − Batch like execution
− ALAP execution
Petri Net simulation ORM synchronization
Fault-tolerant Systems Research Group ICGT '08

12. Budapest University of Technology and Economics 12
Petri net Simulation  Sequential Transition
 Test Case generation reduction
− Inverse property preservation
reduction operation
− Random weighted select
− Few tokens
− Transition/Place ~ 1.1
− Starting from:
 Characteristics
Paradigm
 Execution Features Petri Net
− Random fireable transition LHS size small
selection fan-out small
− 1000 / 1000000 iterations matchings PD
transformation
− Measured:
sequence
Total execution time length Small/ Long
Fault-tolerant Systems Research Group ICGT '08

13. Budapest University of Technology and Economics 13
Object-Relational Mapping: Synchronization
:schemaRef
: Package : Schema
:tableRef
: Class : Table
:columnRef
: Attribute : Attribute : Column : Column
:columnRef
Sync order
orphanTable
1. Orphan schema delete
NEG
2. Orphan Tableclass delete
: tablerRef
C: Class 3. Orphan Tableassoc delete
{DEL} 4. Orphan Columnattr. Delete
OR T : Table 5. Renaming
6. new Classes/Assocs/Columns
A: Association created
: tableRef
Fault-tolerant Systems Research Group ICGT '08

14. Budapest University of Technology and Economics 14
Object-Relational Mapping: Synchronization
 Test Case generation  Source modification
− 1/3 classes deleted
− Fully connected graph
− 1/5 associations deletes
− N classes − ½ attributes renamed
− N(N-1) directed association − 1 new class added and the fully
− K attributes connected graph is rebuilt
 Execution  Characteristic
− Phases Paradigm
Features ORM Syn.
1. Generation
LHS size large
2. Build
fan-out medium
3. Modification
matchings PD
4. Synchronization
transformation
− Measured: sequence
length PD
Synchronization phase
Fault-tolerant Systems Research Group ICGT '08

15. Budapest University of Technology and Economics 15
Talk overview
GT &
Introduction incremental
overview
Case Studies Measurements
Fault-tolerant Systems Research Group ICGT '08

16. Budapest University of Technology and Economics 16
Environment
 Hardware and OS
− 2 GHz Core2
− 2048 MB RAM
− Linux kernel of version 2.6.22
− Sun JVM 1.6.0_05 for VIATRA
− .Net 3.0 on Windows Vista
 Tool related
− VIATRA with and GrGen withouth GUI
− Standard services of the default distribution
Fault-tolerant Systems Research Group ICGT '08

17. Budapest University of Technology and Economics 17
Results of the Petri Net Simulation
•Constant time!
•Predictable runtime
•Comparable with
compiled LS
Fault-tolerant Systems Research Group ICGT '08

18. Budapest University of Technology and Economics 18
Result of the ORM Synchronization
build – RETE
ORM Build and Synchronization sync – RETE
build – LS
100000
sync – LS
10000
1000
Execution time [ms]
100
•Execution time:
10
•Rete: linear
1 •LS: exponential
85 705 1925 7600 in number of 67800
17025 30200 affected element
Model size [N]
Fault-tolerant Systems Research Group ICGT '08

19. Budapest University of Technology and Economics 19
Varró: STS Benchmark
•Non reusable model
VIATRA/RETE
Performance on STS mutex benchmark
elements
•Suites better for LS VIATRA/LS
1000000
GrGen.NET
100000
10000
Execution time (ms)
1000
100
100 200 500 1000 3000 10000 30000
Size of process ring [node count]
Fault-tolerant Systems Research Group ICGT '08

20. Budapest University of Technology and Economics 20
Summary
 Rete based incremental PM
− Increased memory consumption
− Fix overhead
− Predictable linear scaling for practical problems
− Comparable performance with fastest LSs in some cases.
 Contribution to Benchmarking of GTs
− 2 new benchmarks for assessing incremental PM
− Larger models
− ALAP execution
 Different matching strategies for different
− Execution phases
− Patterns in a transformation?
 In overall the performance of GT tools are satisfying.
 Benchmark example descriptions, specifications, and
measurement results available at:
● http://wiki.eclipse.org/VIATRA2/Benchmarks
Fault-tolerant Systems Research Group ICGT '08

21. Budapest University of Technology and Economics 21
:schemaRef
: Package : Schema
:tableRef
: Class : Table
:columnRef
: Attribute : Attribute : Column : Column
:columnRef
orphanTable
NEG
: tablerRef
C: Class
{DEL}
OR T : Table
A: Association
: tableRef
Fault-tolerant Systems Research Group ICGT '08

22. Budapest University of Technology and Economics 22
Fault-tolerant Systems Research Group ICGT '08