Dipartimento di Ingegneria e Scienze
Università degli Studi dell’Aquila
dell’Informazione e Matematica
Mining Metrics for ...
Alfonso Pierantonio
2
Outline
Introduction
Motivation
Measuring metamodels
Calculation of metrics correlation
Selection of...
Alfonso Pierantonio
3
Introduction
Metamodels are a key concept in Model-Driven Engineering:
they represent the «trait-d'u...
Alfonso Pierantonio
4
Motivation
It is of crucial relevance to investigate common characteristics
of metamodels, in order ...
Alfonso Pierantonio
5
Motivation
It is of crucial relevance to investigate common
characteristics of metamodels, in order ...
Analysing metamodel characteristics
by investigating the correlations of
different metrics applied on a corpus of
more tha...
Alfonso Pierantonio
7
Measuring Metamodels
The applied process is able to identify linked
structural characteristics
Under...
Alfonso Pierantonio
8
Metrics calculation
Consists of the application of metrics on a data set of
metamodels
The applied m...
Alfonso Pierantonio
9
Metrics calculation
The corpus of the analyzed metamodels has been
obtained by retrieving metamodels...
Alfonso Pierantonio
10
Metrics Calculation
The metrics calculation has been implemented by
exploiting a model-driven toolc...
Alfonso Pierantonio
11
Metrics Calculation
The metrics calculation has been implemented by
exploiting a model-driven toolc...
Alfonso Pierantonio
12
Metrics Calculation
The metrics calculation has been implemented by
exploiting a model-driven toolc...
Alfonso Pierantonio
13
Metrics Calculation
The metrics calculation has been implemented by
exploiting a model-driven toolc...
Alfonso Pierantonio
14
Calculation of metrics correlations
Correlation is used to detect cross-links and assess
relationsh...
Alfonso Pierantonio
15
Selection of metrics correlations
We have calculated the Pearson’s and Spearman’s
correlation index...
Alfonso Pierantonio
16
Selection of metrics correlations
The bar chart shows the degree of metric correlation:
the higher ...
Alfonso Pierantonio
17
Pearson’s correlation matrix
All the values greater than 0.7 or lesser than -0.73
have been highlig...
Alfonso Pierantonio
18
Pearson’s correlation matrix
All the values greater than 0.7 or lesser than -0.73
have been highlig...
Alfonso Pierantonio
19
Data analysis
The aim is to discuss and interpret the most relevant
correlations between structural...
Alfonso Pierantonio
20
Data analysis: abstraction
How the number of metaclasses is related to the
adoption of abstraction ...
Alfonso Pierantonio
21
Data analysis: abstraction
How the number of metaclasses is related to the
adoption of abstraction ...
Alfonso Pierantonio
22
Data analysis: abstraction
How the number of metaclasses is related to the
adoption of abstraction ...
Alfonso Pierantonio
23
Data analysis: abstraction
How the number of metaclasses is related to the
adoption of abstraction ...
Alfonso Pierantonio
24
Data analysis: abstraction
How the number of metaclasses is related to the
adoption of abstraction ...
Alfonso Pierantonio
25
Metrics
Alfonso Pierantonio
26
Data analysis: abstraction
The #metaclasses and the #classes with supertypes
are strongly correlate...
Alfonso Pierantonio
27
Data analysis: abstraction
This is testified by the graph that shows the values of
the MHS (Max Hie...
Alfonso Pierantonio
28
Data analysis: abstraction
In metamodels with at most 50 metaclasses:
– the number of supertypes in...
Alfonso Pierantonio
29
Data analysis: structural features
How structural features are used with hierarchies
We can conside...
Alfonso Pierantonio
30
Data analysis: structural features
Increasing the #metaclasses with supertypes, the
average # of st...
Alfonso Pierantonio
31
Data analysis: structural features
By considering metamodels having in between 1 and 50
metaclasses...
Alfonso Pierantonio
32
Data analysis: structural features
How the number of featureless metaclasses is
related to hierarch...
Alfonso Pierantonio
33
Data analysis: structural features
How the number of featureless metaclasses is
related to hierarch...
Alfonso Pierantonio
34
Data analysis: structural features
By increasing the number of metaclasses with super
types, the nu...
Alfonso Pierantonio
35
Data analysis: isolated metaclasses
How isolated metaclasses are distributed
In this case we consid...
Alfonso Pierantonio
36
Data analysis: isolated metaclasses
We can claim that they are used only for testing or
educational...
Alfonso Pierantonio
37
Conclusions
We proposed a number of metrics which can be used
to acquire objective, transparent, an...
Alfonso Pierantonio
38
Conclusions
These figures have been discussed in detail
highlighting the followings
- the adoption ...
Alfonso Pierantonio
39
Conclusions
These figures have been discussed in detail
highlighting the following
Thank you
Upcoming SlideShare
Loading in...5
×

Mining Metrics for Understanding Metamodel Characteristics

178

Published on

Presentation at MiSE 2014 @ ICSE in Hyderabad, India
https://sselab.de/lab2/public/wiki/MiSE/index.php?title=Main_Page

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
178
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Mining Metrics for Understanding Metamodel Characteristics"

  1. 1. Dipartimento di Ingegneria e Scienze Università degli Studi dell’Aquila dell’Informazione e Matematica Mining Metrics for Understanding Metamodel Characteristics Alfonso Pierantonio Juri Di Rocco Davide Di Ruscio Ludovico Iovino
  2. 2. Alfonso Pierantonio 2 Outline Introduction Motivation Measuring metamodels Calculation of metrics correlation Selection of metrics correlation Data Analisys Conclusions Future works
  3. 3. Alfonso Pierantonio 3 Introduction Metamodels are a key concept in Model-Driven Engineering: they represent the «trait-d'union» among all constituent components Metamodels formally define the modeling primitives used in modeling activities
  4. 4. Alfonso Pierantonio 4 Motivation It is of crucial relevance to investigate common characteristics of metamodels, in order to have a better understanding on – how they evolve over time – what is the impact of metamodel changes throughout the modeling ecosystem Metamodel
  5. 5. Alfonso Pierantonio 5 Motivation It is of crucial relevance to investigate common characteristics of metamodels, in order to have a better understanding on – how they evolve over time – what is the impact of metamodel changes throughout the modeling ecosystem Analyse metamodels in order to evalutate their structural characteristics and the impact they might have during the whole metamodel life-cycle especially in case of metamodel evolutions
  6. 6. Analysing metamodel characteristics by investigating the correlations of different metrics applied on a corpus of more than 450 metamodels
  7. 7. Alfonso Pierantonio 7 Measuring Metamodels The applied process is able to identify linked structural characteristics Understand how they might change depending on the nature of metamodels
  8. 8. Alfonso Pierantonio 8 Metrics calculation Consists of the application of metrics on a data set of metamodels The applied metrics are borrowed from [1] and we added new ones by leading to a set of 28 metrics An excerpt of the considered metrics have been considered in rest of the presentation [1] W. James, Z. Athansios, M. Nicholas, R. Louis, K. Dimitios, P. Richard, and P. Fiona. What do metamodels really look like? Frontiers of Computer Science, 2013.
  9. 9. Alfonso Pierantonio 9 Metrics calculation The corpus of the analyzed metamodels has been obtained by retrieving metamodels from different repositories, i.e., EMFText Zoo, ATLZoo, Github, GoogleCode For a total number of 466 metamodels belonging to different technical spaces and domains
  10. 10. Alfonso Pierantonio 10 Metrics Calculation The metrics calculation has been implemented by exploiting a model-driven toolchain
  11. 11. Alfonso Pierantonio 11 Metrics Calculation The metrics calculation has been implemented by exploiting a model-driven toolchainThe Metamodel Metrics Calculator is able to calculate for each metamodel all the considered metrics
  12. 12. Alfonso Pierantonio 12 Metrics Calculation The metrics calculation has been implemented by exploiting a model-driven toolchain The Metamodel Metrics Calculator is an ATL transformation whose target models conform to the Metrics metamodel
  13. 13. Alfonso Pierantonio 13 Metrics Calculation The metrics calculation has been implemented by exploiting a model-driven toolchain Generating CVS files enables the adoption of statistical tools like IBM SPSS, Microsoft Excel, and Libreoffice Calc for subsequent analysis of the generated data
  14. 14. Alfonso Pierantonio 14 Calculation of metrics correlations Correlation is used to detect cross-links and assess relationships among observed data We have considered Pearson’s and Spearman’s coefficients to measure the correlations among calculated metamodel metrics Spearman only for highlighting curvilinear correlations Both Pearson’s and Spearman’s correlation indexes assume values in the range of -1.00 (perfect negative correlation) and +1.00 (perfect positive correlation) A correlation with value 0 indicates that between two variables there is no correlation
  15. 15. Alfonso Pierantonio 15 Selection of metrics correlations We have calculated the Pearson’s and Spearman’s correlation indexes for all the values of the considered metrics For each couple we have selected the coefficient index (between Pearson and Spearman) having higher correlation values
  16. 16. Alfonso Pierantonio 16 Selection of metrics correlations The bar chart shows the degree of metric correlation: the higher the bar, the more the metric is correlated with other metrics.
  17. 17. Alfonso Pierantonio 17 Pearson’s correlation matrix All the values greater than 0.7 or lesser than -0.73 have been highlighted to select the metrics that are most related according to the Pearson’s index.
  18. 18. Alfonso Pierantonio 18 Pearson’s correlation matrix All the values greater than 0.7 or lesser than -0.73 have been highlighted to select the metrics that are most related according to the Pearson’s index.
  19. 19. Alfonso Pierantonio 19 Data analysis The aim is to discuss and interpret the most relevant correlations between structural characteristics which have been found in the previous stages
  20. 20. Alfonso Pierantonio 20 Data analysis: abstraction How the number of metaclasses is related to the adoption of abstraction constructs, ie. abstract metaclasses and supertypes
  21. 21. Alfonso Pierantonio 21 Data analysis: abstraction How the number of metaclasses is related to the adoption of abstraction constructs, ie. abstract metaclasses and supertypes
  22. 22. Alfonso Pierantonio 22 Data analysis: abstraction How the number of metaclasses is related to the adoption of abstraction constructs, ie. abstract metaclasses and supertypes MGHL Maximum hierarchical depth
  23. 23. Alfonso Pierantonio 23 Data analysis: abstraction How the number of metaclasses is related to the adoption of abstraction constructs, ie. abstract metaclasses and supertypes MHS Maximum Hierarchy Siblings
  24. 24. Alfonso Pierantonio 24 Data analysis: abstraction How the number of metaclasses is related to the adoption of abstraction constructs, ie. abstract metaclasses and supertypes MTCWS Number of metaclasses having at least one super type
  25. 25. Alfonso Pierantonio 25 Metrics
  26. 26. Alfonso Pierantonio 26 Data analysis: abstraction The #metaclasses and the #classes with supertypes are strongly correlated (with Pearson index 0.99) When the metamodel grows in size, the number of inheritance hierarchies also increases, however the hierarchy depth is somehow independent from the metamodel size. Interestingly, metamodel designers prefer to add siblings in hierarchies instead of adding new hierarchy levels
  27. 27. Alfonso Pierantonio 27 Data analysis: abstraction This is testified by the graph that shows the values of the MHS (Max Hierarchy Sibling) and MGHL (Max generalization hierarchical level) metrics Confirmed by the Pearson correlation indexes between MC and MHS (0.70) and that between MC and MGHL (0.66).
  28. 28. Alfonso Pierantonio 28 Data analysis: abstraction In metamodels with at most 50 metaclasses: – the number of supertypes in hierarchy is in between 0 and 20 – the number of siblings in a hierarchy is in between 0 and 10 – the maximum height of a hierarchy is in between 0 and 5
  29. 29. Alfonso Pierantonio 29 Data analysis: structural features How structural features are used with hierarchies We can consider the average number of features (ASF) and the total number of metaclasses with supertypes (MCWS) metrics The Spearman approach permits to identify a greater correlation index
  30. 30. Alfonso Pierantonio 30 Data analysis: structural features Increasing the #metaclasses with supertypes, the average # of structural features in a metaclass decreases
  31. 31. Alfonso Pierantonio 31 Data analysis: structural features By considering metamodels having in between 1 and 50 metaclasses, the average number of features (excluding the inherited ones) of a metaclass ranges between 1 and 5
  32. 32. Alfonso Pierantonio 32 Data analysis: structural features How the number of featureless metaclasses is related to hierarchies height This can indicate how specializations of metaclasses can introduce or reduce structural features in metamodels MCWS and IFLMC are strongly correlated as supported by the Pearson’s index having value 0.890
  33. 33. Alfonso Pierantonio 33 Data analysis: structural features How the number of featureless metaclasses is related to hierarchies height
  34. 34. Alfonso Pierantonio 34 Data analysis: structural features By increasing the number of metaclasses with super types, the number of metaclasses without attributes or references increases too This means that when hierarchies are introduced, usually existing features are subject to refactoring operations mainly to move them to super classes and to create leaves in the hierarchies inheriting features from the super types.
  35. 35. Alfonso Pierantonio 35 Data analysis: isolated metaclasses How isolated metaclasses are distributed In this case we considered another statistical instrument named Pareto analysis, which is commonly referred to as the 80-20 principle (20% of the causes account for 80% of the defects) About 80% of the analyzed metamodels have a percentage of isolated metaclasses in the range 0- 19%, by testifying the fact that isolated metaclasses are not commonly used
  36. 36. Alfonso Pierantonio 36 Data analysis: isolated metaclasses We can claim that they are used only for testing or educational purposes
  37. 37. Alfonso Pierantonio 37 Conclusions We proposed a number of metrics which can be used to acquire objective, transparent, and reproducible measurements of metamodels. The major goal is to better understand the main characteristic of metamodels, how they are coupled, and how they change depending on the metamodel structure A correlation analysis has been performed to identify the most cross-linked metrics, which have, in turn, been computed over 450 metamodels
  38. 38. Alfonso Pierantonio 38 Conclusions These figures have been discussed in detail highlighting the followings - the adoption of inheritance is proportional to the size of metamodels - the number of metaclasses with supertypes are inversely proportional to the average number of structural features - the number of metaclasses with supertypes is proportional to the number of metaclasses without attributes or references - isolated metaclasses are not commonly used apart from testing or educational purposes.
  39. 39. Alfonso Pierantonio 39 Conclusions These figures have been discussed in detail highlighting the following
  40. 40. Thank you
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×