In MDE resolving pragmatic issues related to the management of models is key to success. Model comparison is one of the most challenging operations playing a central role in a wide range of modelling activities including model versioning, evolution and even collaborative and distributed specification of models. Over the last decade, several syntactic methods have been proposed to compare models even though they struggle in achieving higher levels of accuracy especially when the semantics of the application domain has to be considered. Existing methods improve comparison precision at the price of high performance costs. In this talk I presented a lightweight semantic comparison method, which relies on a new matching algorithm that considers ontological information encoded in the WordNet lexical database further than ordinary syntactical and structural correlations. The approach has been implemented as extension of EMFCompare and evaluated to measure its precision and performances when compared to existing approaches.

1. Dipartimento di Ingegneria e Scienze
Università degli Studi dell’Aquila
dell’Informazione e Matematica
Semantic-based Model Matching with
EMFCompare
Davide Di Ruscio
davide.diruscio@univaq.it
@ddiruscio
Models and Evolution Workshop at MoDELS 2016 – October 2, 2016 – Saint-Malo, France

2. ME‘16 – October 2, 2016 – Saint-Malo, France
2
Joint work with
Alfonso Pierantonio
Unversity of L’Aquila
(Italy)
Ludovico Iovino
Gran Sasso Science Institute
(Italy)
Juri Di Rocco
Unversity of L’Aquila
(Italy)
Lorenzo Addazi
Malardalen University
(Sweden)
Antonio Cicchetti
Malardalen University
(Sweden)

3. ME‘16 – October 2, 2016 – Saint-Malo, France
3
Introduction
Model comparison is one of the most challenging
operations in MDE
It underpins a wide range of modelling activities
• E.g., model versioning, evolution, collaborative modeling, …
Calculating model differences relies on the model
matching problem
• It can be reduced to the problem of finding correspondences between
two given graphs (Graph Isomorphism Problem, NP-Hard)

4. ME‘16 – October 2, 2016 – Saint-Malo, France
4
Introduction
a
b
f
c
e
d
Version 1
a
k
l
c
e
dVersion 2
m

5. ME‘16 – October 2, 2016 – Saint-Malo, France
5
Introduction
a
b
f
c
e
d
Version 1
a
k
l
c
e
dVersion 2
m
Establish
correspondences
Calculate
differences

6. ME‘16 – October 2, 2016 – Saint-Malo, France
6
Introduction
a
b
f
c
e
d
Version 1
a
k
l
c
e
dVersion 2
m
Establish
correspondences
Calculate
differences

7. ME‘16 – October 2, 2016 – Saint-Malo, France
7
Introduction
a
b
f
c
e
d
Version 1
a
k
l
c
e
dVersion 2
m
Establish
correspondences
Calculate
differences
> Rename node b as k
> Rename node f as l
> Add node m
> Add edge from k to m

8. ME‘16 – October 2, 2016 – Saint-Malo, France
8
Model-matching
Static Identity-Based Matching: each model element
has a persistent unique identifier that is assigned to it
upon creation
Signature-Based Matching: the identifier of each model
element is dynamically calculated by combining the
values of its features
Similarity-Based Matching: models are typed attribute
graphs and matching elements are identified by
considering the aggregated similarity of their features.
Language-Specific Matching: matching algorithms are
tailored to a particular modelling language

9. ME‘16 – October 2, 2016 – Saint-Malo, France
9
Similiartiy-based matching
Extensible
• Static identity-based or signature-based matching can be also added
by defining custom generator functions

10. ME‘16 – October 2, 2016 – Saint-Malo, France
10
The default match engine
The Levenshtein distance algorithm is applied on the
string representation of the elements
• For optimisation purposes the models are compared by considering
elements selected within a proper search window
...
foreach (elM1 : Model1.getElements())
foreach (elM2 : elM1.getWindowElements())
result[elM1][elM2] = calculateSimilarity(elM1, elM2)
return createMatches(result)
...

11. ME‘16 – October 2, 2016 – Saint-Malo, France
11
A meta-model evolution scenario
A University theses management metamodel

12. ME‘16 – October 2, 2016 – Saint-Malo, France
12
A meta-model evolution scenario
A University theses management metamodel
Extract super class

13. ME‘16 – October 2, 2016 – Saint-Malo, France
13
A meta-model evolution scenario
A University theses management metamodel
Attribute renaming

14. ME‘16 – October 2, 2016 – Saint-Malo, France
14
A meta-model evolution scenario
A University theses management metamodel

15. ME‘16 – October 2, 2016 – Saint-Malo, France
15
A meta-model evolution scenario
A University theses management metamodel

16. ME‘16 – October 2, 2016 – Saint-Malo, France
16
A meta-model evolution scenario
A University theses management metamodel

17. ME‘16 – October 2, 2016 – Saint-Malo, France
17
Contextual issues: limited consideration of the
features characterising the elements
surrounding/containing the compared one
Linguistic issues: lack of semantical evaluation of
the features characterizing the compared elements
• False-negative e.g., renaming a given class using a syntactically
different name
• False-positive e.g., renaming a given class using a semantically
different term, which however presents a strong syntactical
similarity

18. ME‘16 – October 2, 2016 – Saint-Malo, France
19
Proposed approach
Semantic Match Engine
• Use of the WordNet lexical dictionary as ontological source

19. ME‘16 – October 2, 2016 – Saint-Malo, France
20
WordNet in a nutshell
Lexical database for the English language
English words are grouped into sets of synonyms
(synsets)
Each synset includes
- a generic definition joining the contained words
- semantic relationships connecting it to other synsets
http://www.cs.princeton.edu/courses/archive/fall16/cos226/assignments/wordnet.html

20. ME‘16 – October 2, 2016 – Saint-Malo, France
21
WordNet in a nutshell
Lexical database for the English language
English words are grouped into sets of synonyms
(synsets)
Each synset includes
- a generic definition joining the contained words
- semantic relationships connecting it to other synsets
http://www.cs.princeton.edu/courses/archive/fall16/cos226/assignments/wordnet.html

21. ME‘16 – October 2, 2016 – Saint-Malo, France
22
The proposed
semantic model matching
function createMatches(Comparison comparison, List
leftEObjects, List rightEObjects){
SemanticMatch root = createSemanticMatch(null, null);
exploreMatches(root, leftEObjects, rightEObjects);
evaluateMatches(root);
filterMatches(root, comparison);
}
Exploration
Evaluation
Filtering

22. ME‘16 – October 2, 2016 – Saint-Malo, France
23
The proposed
semantic model matching
function createMatches(Comparison comparison, List
leftEObjects, List rightEObjects){
SemanticMatch root = createSemanticMatch(null, null);
exploreMatches(root, leftEObjects, rightEObjects);
evaluateMatches(root);
filterMatches(root, comparison);
}
Exploration
Evaluation
Filtering
A labelled graph representation of the
compared models is produced
• each node represents a semantic match
• each incoming or outgoing labelled edge
represents a connection with its parents or
children elements

23. ME‘16 – October 2, 2016 – Saint-Malo, France
24
Type: EAttribute
Source: Student.username
Target: User.password
Sim: null
Type: EClass
Source: Student
Target: User
Sim: null
Type: EClass
Source: Student
Target: Student
Sim: null
Type: EAttribute
Source: Student.password
Target:User.password
Sim: nullType: EAttribute
Source: Student.password
Target: User.username
Sim: nullType: EAttribute
Source: Student.username
Target: User.username
Sim: null

24. ME‘16 – October 2, 2016 – Saint-Malo, France
25
The proposed
semantic model matching
function createMatches(Comparison comparison, List
leftEObjects, List rightEObjects){
SemanticMatch root = createSemanticMatch(null, null);
exploreMatches(root, leftEObjects, rightEObjects);
evaluateMatches(root);
filterMatches(root, comparison);
}
Exploration
Evaluation
Filtering
Each SemantichMatch node is integrated
with the semantic distance value between
the encapsulated element

25. ME‘16 – October 2, 2016 – Saint-Malo, France
26
Type: EAttribute
Source: Student.username
Target: User.password
Sim: 0.2
Type: EClass
Source: Student
Target: User
Sim: 0.4
Type: EClass
Source: Student
Target: Student
Sim: 1
Type: EAttribute
Source: Student.password
Target:User.password
Sim: 0.6Type: EAttribute
Source: Student.password
Target: User.username
Sim: 0.2Type: EAttribute
Source: Student.username
Target: User.username
Sim: 0.6

26. ME‘16 – October 2, 2016 – Saint-Malo, France
27
The proposed
semantic model matching
function createMatches(Comparison comparison, List
leftEObjects, List rightEObjects){
SemanticMatch root = createSemanticMatch(null, null);
exploreMatches(root, leftEObjects, rightEObjects);
evaluateMatches(root);
filterMatches(root, comparison);
}
Exploration
Evaluation
Filtering
The set of SemanticMatch elements are
filtered out with respect to a predefined
threshold

27. ME‘16 – October 2, 2016 – Saint-Malo, France
28
Type: EAttribute
Source: Student.username
Target: User.password
Sim: 0.2
Type: EClass
Source: Student
Target: User
Sim: 0.4
Type: EClass
Source: Student
Target: Student
Sim: 1
Type: EAttribute
Source: Student.password
Target:User.password
Sim: 0.6Type: EAttribute
ource: Student.password
Target: User.username
Sim: 0.2Type: EAttribute
Source: Student.username
Target: User.username
Sim: 0.6

28. ME‘16 – October 2, 2016 – Saint-Malo, France
29
Experiments
The Model Exchange Benchmark
• 5 structural modelling languages
• All the possible pairs of metamodels are given as input to:
• Semantic EMFCompare
• EMFCompare
• GAMMA(*)
• Coma++, FOAM, Crosi, Alignment API, AMW
(*) M. Kessentini, A. Ouni, P. Langer, M. Wimmer, and S. Bechikh, “Search-based metamodel matching
with structural and syntactic measures,” J. Syst. Softw., vol. 97, no. C, pp. 1–14, Oct. 2014.

29. ME‘16 – October 2, 2016 – Saint-Malo, France
30
Experiments
Measures

30. ME‘16 – October 2, 2016 – Saint-Malo, France
31
Experiments
Measures
It denotes the percentage of
correctly matched elements
with respect to all the
proposed matches

31. ME‘16 – October 2, 2016 – Saint-Malo, France
32
Experiments
Measures
It denotes the percentage of
correctly matched elements
with respect to all the
expected matches

32. ME‘16 – October 2, 2016 – Saint-Malo, France
33
Experiments
Measures It combines Precision and
Recall to get an equally
weighted average value of
the measures

33. ME‘16 – October 2, 2016 – Saint-Malo, France
34
Experiments
GAMMA provides best results with respect to Precision,
Recall, and F-Measure
GAMMA uses SBSE approaches and it requires to be
initialized with a set of initial solutions (knowledge base)

34. ME‘16 – October 2, 2016 – Saint-Malo, France
35
Experiments
Semantic EMFCompare:
• produces more matches than expected
• in some cases has lower Precision than EMFCompare
• only in one case F-Measure is lower than EMFCompare

35. ME‘16 – October 2, 2016 – Saint-Malo, France
36
Experiments

36. ME‘16 – October 2, 2016 – Saint-Malo, France
37
Lessons learnt
Extending EMFCompare with semantic aspects can be
done in a lightweight manner
An increasing matching power can come at the price of
an increasing imprecision (more false-positives and false-
negatives)
The selection of the appropriate dictionary (depending on
the artifacts to be compared) can make the difference
• Comparing metamodels is semantically different than comparing models
of specific domains
Performing experiments can be an issue due to the lack
of models to be used as test cases
• Existing model mutations approaches should be extended to implement
“semantics-aware” mutations

37. ME‘16 – October 2, 2016 – Saint-Malo, France
38
Conclusion and Future Work
Model comparison is a very complex task
It underpins the management of a wide number of
(meta-)model (co-)evolution scenarios
An extension of the EMFCompare tool has been
proposed to enable “semantics-aware” matches
Further experiments will be performed by considering
the application of different dictionaries depending on
the kinds of artifacts to be matched

38. ME‘16 – October 2, 2016 – Saint-Malo, France
39