This document describes an empirical study that compared the effectiveness of two variability management approaches for software product lines (SPLs) at the UML class level: PLUS and SMarty. The study found that PLUS was more effective at identifying and representing variabilities in class models. Based on participant feedback, guidelines were improved for SMarty and a new experiment is planned to evaluate the updated SMarty approach against PLUS. The results provide evidence that PLUS is currently more effective but further studies are needed to generalize findings and potentially improve SMarty's effectiveness.
Empirically Based Evolution of Variability Management Approach for UML Class Models
1. Empirically Based Evolution of a
Variability Management Approach
at UML Class Level
Anderson Marcolino, M.Sc.
State University of Maringá (DIN-UEM) – Brazil
Edson OliveiraJr, Ph.D.
edson@din.uem.br
State University of Maringá (DIN-UEM) - Brazil
Itana Gimenes, Ph.D.
State University of Maringá (DIN-UEM) – Brazil
Ellen Barbosa, Ph.D.
University of São Paulo (ICMC-USP) – Brazil
2. 2
Agenda
Introduction
Background
SPL and Variability Management
The Gomaa Method
The SMarty Approach
SMarty Evolution based on Empirical Study
Conclusion and Future Works
3. 3
Introduction
Software Product Line (SPL) is a consolidated reuse technique,
in which several products share similar features and
variabilities
allowing mass customization for specific market needs
Variability represents how such products differ one another,
thus
Variability Management (VM) is a key issue for the success of SPLs
Literature presents several well-known approaches for VM,
especially object-oriented and UML-based.
However, their effectiveness was not experimentally analyzed, which can
make technology transfer feasible
4. 4
Introduction
The SPL approach encompasses three main
activities:
Domain Engineering
Application Engineering
Management (Organizational and Technical)
Variability management is one of the most important
SPL management activities
5. 5
Introduction
Four main concepts are taken into consideration
for VM:
Variability, Variation Point and Variants
PLUS and SMarty are two well-known
approaches for VM in UML-based SPLs
However, PLUS has been taken as a basis for several UML-
based SPL projects
6. 6
Objectives of this work
1. Comparing the effectiveness of SMarty and
PLUS with regard to the identification and
representation of variability in UML classes
2. Allowing the SMarty evolution in order to
increase its effectiveness against PLUS
7. 7
Background - PLUS
The PLUS method
It encompasses a variability management activity with regard to
use cases and classes, using stereotypes with no guidelines to
apply them / Domain experts must use their knowledge for VM /
Variant constraints are not available.
Use Case?
<<kernel>> Used to represent mandatory elements. Yes.
<<optional>>
Used to represent optional elements. It represents an
element that can be selected or not in a specific product. Yes.
<<alternative>>
Used to represent alternative elements, mutually exclusive
elements. Yes.
Stereotype Description
Does
Use
cases
only!!!
Use
cases
only!!!
8. 8
Background - PLUS
An example of the PLUS variability representation for
classes:
types
<<kernel>>
SortingElement
<<optional>>
NumericElement
<<optional>>
StringElement
-sortingElements
1..*
9. 9
Background - SMarty
Stereotype-based Management of Variability
(SMarty) approach
It is composed of:
an UML 2 profile, the SMartyProfile with stereotypes
for VM in use case, class, component, sequence, and
activity diagrams; and
a process, the SMartyProcess, with guidelines to
support variability identification and representation
10. 10
Background - SMarty
SMartyProfile comprises the following stereotypes, which
can be applied to UML models:
<<variability>> represents the concept of PL variability;
<<variant>> this abstract stereotype is specialized in four other non-
abstract stereotypes which are: <<mandatory>>, <<optional>>,
<<alternative_OR>>, and <<alternative_XOR>>;
<<mutex>> is a mutually exclusive relationship between two variants;
and
<<requires>> is a relationship between two variants in which the
selected variant requires the presence of another specific variant.
11. 11
Background - SMarty
An example of the
SMarty variability
representation for
classes:
types
<<madatory>>
<<variationpoint>>
SortingElement
<<alternative_OR>>
NumericElement
<<alternative_OR>>
StringElement
-sortingElements
<<comment>>
{allow sAddingVar = false,
bindingTime = DESIGN_TIME,
maxSelection = 2,
minSelection = 1,
name = "sorting element",
variants = "NumericElement, StringElement"}
1..*
12. 12
The Experimental Study
Aim (Basili’s GQM template)
Compare PLUS and SMarty, for the purpose of characterize
the most effectiveness, with respect to the capability of
identification and representation of variabilities in Software
Product Line class models, from the point of view of software
product line architects, in the context of master and Ph.D.
students of the Software Engineering area from the University of
São Paulo (ICMC/USP) and Federal University of São Carlos
(UFSCar).
14. 14
The Experimental Study
Planning
Hypotheses Formulation
H0 : µ (effectiveness(X)) = µ (effectiveness(Y));
H1 : µ (effectiveness(X)) < µ (effectiveness(Y)); and
H2 : µ (effectiveness(X)) > µ (effectiveness(Y)).
Where X = PLUS and Y = SMarty.
15. 15
The Experimental Study
Planning
Dependent Variables: the effectiveness calculated for each
variability management approach (X and Y) as follows:
16. 16
The Experimental Study
Planning
Independent Variables:
the variability management approach, a factor with
two treatments (X and Y); and
the SPL, a factor with two treatments (E-commerce
and AGM SPLs).
18. 18
The Experimental Study
Effectiveness of the Approaches
Collected Data Normality Test: the Shapiro-Wilk normality test
was applied to the E-commerce and AGM samples providing the
following results:
sample X was considered non-normal and
sample Y was considered normal.
19. 19
The Experimental Study
Box Plot of Effectiveness (Class)
Spreadsheet1 10v*12c
Median; Box: 25%-75%; Whisker: Non-Outlier Range
Median
25%-75%
Non-Outlier Range
Outliers
Extremes
Effectiveness Y Approach.
Effectiveness X Approach.
-5
0
5
10
15
20
25
30
20. 20
The Experimental Study
Effectiveness of the Approaches
Mann-Whitney-Wilcoxon for SampleX and SampleY:
Detailed explanation on samples is in the paper
There is evidence that the X approach (PLUS) is more effective in
identifying and representing variability in class models than the Y
approach (SMarty)
This result corroborates to reject the null hypothesis (H0)
of this study: PLUS is more effective than SMarty for
SPLs class models and subjects taken into account.
21. 21
The Experimental Study
Threats to Validity
Conclusion Validity
Sample size is a major concern, which must be increased in
prospective studies
Random capacity was not applied to subjects selection, thus
generalization of results could not be inferred
Construct Validity
Independent variable variability modeling approach was
guaranteed by the pilot project undertaken
22. 22
The Experimental Study
Threats to Validity
Internal Validity
Differences among subjects – variations on skills were reduced by
performing training session and tasks in the same order
Fatigue effects – on average the experiment took 80 minutes, thus
fatigue was not considered a problem for the study
Influence among subjects – could not be really controlled. They took
the experiment under the supervision of a human observer.
23. 23
The Experimental Study
Threats to Validity
External Validity
Instrumentation – failing to use real class models, as the e-commerce
and AGM are not commercial SPLs. More experiments must be
carried out with real SPLs.
Subjects – we could take advantages of the benefits of taking into
account students for performing experiments as pointed out by
Carver et al. (2003)
24. 24
Empirical Evolution of SMarty
Improvements for the SMarty approach based on the
feedback of the subjects:
E-commerce SPL
Subjects reported difficulties on the application of SMarty to the e-
commerce SPL class model as it has no elements of class modeling,
such as, inheritance, aggregation, and generalization E-
commerce was analyzed to improve SMarty
Training session
Subjects indicated that they need more time for training and
application of SMarty
25. 25
Empirical Evolution of SMarty
Improvements for the SMarty approach based on the
feedback of the subjects:
Amount of stereotypes
Subjects questioned the difference between PLUS and SMarty as
the later has several more stereotypes
Arrangement of the SPL models:
E-commerce class models are mode complex than AGM
E-commerce models were the first models given to the subjects
26. 26
Empirical Evolution of SMarty
Based on such subjects feedback, the following
improvements were made:
New guidelines were added to SMarty, encompassing the level of
abstraction and elements of the e-commerce class models
CL2. Class models elements, related with associations in which their
attributes have aggregationKind as none, or, do not represent
aggregation or composition, suggest mandatory or optional variants.
CL2.1. For the identification of possible optional variants classes
related through associations where the multiplicity in one of the ends
of an association, to each object class found at the opposite end
match * (zero or more) or 0..n where n is any integer different from
zero; suggest an optional class
27. 27
Empirical Evolution of SMarty
Based on such subjects feedback, the following
improvements were made:
We realized that with a few less stereotypes, PLUS is easier to
apply. However, such an “easibility” might jeopardize the
generation of specific products as we can observe ambiguity in
PLUS class models
28. 28
Revisiting objectives of this work…
1. Comparing the effectiveness of SMarty and PLUS with
regard to the identification and representation of
variability in UML classes
PLUS is more effective
1. Allowing the SMarty evolution in order to increase its
effectiveness against PLUS
New guidelines for class models need a new experiment
Reduce stereotypes or avoid ambiguity during specific products
generation? New experiment is being carried out
29. 29
Conclusion
Industry needs that the scientific community tests existing and new
technologies, such as SMarty, identifying their effectiveness to make
technology transfer easier and reliable.
The experimental study presented in this paper demonstrates the
ability to use variability management approaches. Their
effectiveness were analyzed by modeling variability in class models
of two SPLs.
Empirical based improvements could be made to SMarty in order to
incorporate support for different class model elements by means of
new guidelines
30. 30
Conclusion
Shapiro-Wilk normality test was applied to both samples
Mann-Whitney-Wilcoxon test analyzed the effectiveness of PLUS
and SMarty
Obtained results provide evidence that the PLUS is more effective
than SMarty for modeling variability in UML class models, taking into
account the E-commerce and the AGM SPLs for this study.
31. 31
Future Works
New experimental studies and replications must be planned and
conducted to make it possible to reduce the threats, increasing the
effectiveness of SMarty towards generalizing the results.
As new experiments, we are:
planning a new experiment for characterize the improved version of
SMarty effective against PLUS; and
planning an experiment to analyze the effectiveness of PLUS and
SMarty and their impact on the generation of specific products.