This document describes an empirical study that compared the effectiveness of two variability management approaches for software product lines (SPLs) at the UML class level: PLUS and SMarty. The study found that PLUS was more effective at identifying and representing variabilities in class models. Based on participant feedback, guidelines were improved for SMarty and a new experiment is planned to evaluate the updated SMarty approach against PLUS. The results provide evidence that PLUS is currently more effective but further studies are needed to generalize findings and potentially improve SMarty's effectiveness.

1. Empirically Based Evolution of a
Variability Management Approach
at UML Class Level
Anderson Marcolino, M.Sc.
State University of Maringá (DIN-UEM) – Brazil
Edson OliveiraJr, Ph.D.
edson@din.uem.br
State University of Maringá (DIN-UEM) - Brazil
Itana Gimenes, Ph.D.
State University of Maringá (DIN-UEM) – Brazil
Ellen Barbosa, Ph.D.
University of São Paulo (ICMC-USP) – Brazil

2. 2
Agenda
 Introduction
 Background
 SPL and Variability Management
 The Gomaa Method
 The SMarty Approach
 SMarty Evolution based on Empirical Study
 Conclusion and Future Works

3. 3
Introduction
 Software Product Line (SPL) is a consolidated reuse technique,
in which several products share similar features and
variabilities
 allowing mass customization for specific market needs
 Variability represents how such products differ one another,
thus
 Variability Management (VM) is a key issue for the success of SPLs
 Literature presents several well-known approaches for VM,
especially object-oriented and UML-based.
 However, their effectiveness was not experimentally analyzed, which can
make technology transfer feasible

4. 4
Introduction
 The SPL approach encompasses three main
activities:
 Domain Engineering
 Application Engineering
 Management (Organizational and Technical)
 Variability management is one of the most important
SPL management activities

5. 5
Introduction
 Four main concepts are taken into consideration
for VM:
 Variability, Variation Point and Variants
 PLUS and SMarty are two well-known
approaches for VM in UML-based SPLs
 However, PLUS has been taken as a basis for several UML-
based SPL projects

6. 6
Objectives of this work
1. Comparing the effectiveness of SMarty and
PLUS with regard to the identification and
representation of variability in UML classes
2. Allowing the SMarty evolution in order to
increase its effectiveness against PLUS

7. 7
Background - PLUS
 The PLUS method
 It encompasses a variability management activity with regard to
use cases and classes, using stereotypes with no guidelines to
apply them / Domain experts must use their knowledge for VM /
Variant constraints are not available.
Use Case?
<<kernel>> Used to represent mandatory elements. Yes.
<<optional>>
Used to represent optional elements. It represents an
element that can be selected or not in a specific product. Yes.
<<alternative>>
Used to represent alternative elements, mutually exclusive
elements. Yes.
Stereotype Description
Does
Use
cases
only!!!
Use
cases
only!!!

8. 8
Background - PLUS
 An example of the PLUS variability representation for
classes:
types
<<kernel>>
SortingElement
<<optional>>
NumericElement
<<optional>>
StringElement
-sortingElements
1..*

9. 9
Background - SMarty
 Stereotype-based Management of Variability
(SMarty) approach
 It is composed of:
 an UML 2 profile, the SMartyProfile with stereotypes
for VM in use case, class, component, sequence, and
activity diagrams; and
 a process, the SMartyProcess, with guidelines to
support variability identification and representation

10. 10
Background - SMarty
 SMartyProfile comprises the following stereotypes, which
can be applied to UML models:
 <<variability>> represents the concept of PL variability;
 <<variant>> this abstract stereotype is specialized in four other non-
abstract stereotypes which are: <<mandatory>>, <<optional>>,
<<alternative_OR>>, and <<alternative_XOR>>;
 <<mutex>> is a mutually exclusive relationship between two variants;
and
 <<requires>> is a relationship between two variants in which the
selected variant requires the presence of another specific variant.

11. 11
Background - SMarty
 An example of the
SMarty variability
representation for
classes:
types
<<madatory>>
<<variationpoint>>
SortingElement
<<alternative_OR>>
NumericElement
<<alternative_OR>>
StringElement
-sortingElements
<<comment>>
{allow sAddingVar = false,
bindingTime = DESIGN_TIME,
maxSelection = 2,
minSelection = 1,
name = "sorting element",
variants = "NumericElement, StringElement"}
1..*

12. 12
The Experimental Study
 Aim (Basili’s GQM template)
 Compare PLUS and SMarty, for the purpose of characterize
the most effectiveness, with respect to the capability of
identification and representation of variabilities in Software
Product Line class models, from the point of view of software
product line architects, in the context of master and Ph.D.
students of the Software Engineering area from the University of
São Paulo (ICMC/USP) and Federal University of São Carlos
(UFSCar).

13. 13
The Experimental Study
 Planning
 Pilot Project
 Selection of Subjects
 Training

14. 14
The Experimental Study
 Planning
 Hypotheses Formulation
 H0 : µ (effectiveness(X)) = µ (effectiveness(Y));
 H1 : µ (effectiveness(X)) < µ (effectiveness(Y)); and
 H2 : µ (effectiveness(X)) > µ (effectiveness(Y)).
 Where X = PLUS and Y = SMarty.

15. 15
The Experimental Study
 Planning
 Dependent Variables: the effectiveness calculated for each
variability management approach (X and Y) as follows:

16. 16
The Experimental Study
 Planning
 Independent Variables:
 the variability management approach, a factor with
two treatments (X and Y); and
 the SPL, a factor with two treatments (E-commerce
and AGM SPLs).

17. 17
The Experimental Study
Execution
 Instrumentation
 Participation Procedure

18. 18
The Experimental Study
 Effectiveness of the Approaches
 Collected Data Normality Test: the Shapiro-Wilk normality test
was applied to the E-commerce and AGM samples providing the
following results:
 sample X was considered non-normal and
sample Y was considered normal.

19. 19
The Experimental Study
Box Plot of Effectiveness (Class)
Spreadsheet1 10v*12c
Median; Box: 25%-75%; Whisker: Non-Outlier Range
Median
25%-75%
Non-Outlier Range
Outliers
Extremes
Effectiveness Y Approach.
Effectiveness X Approach.
-5
0
5
10
15
20
25
30

20. 20
The Experimental Study
 Effectiveness of the Approaches
 Mann-Whitney-Wilcoxon for SampleX and SampleY:
 Detailed explanation on samples is in the paper
 There is evidence that the X approach (PLUS) is more effective in
identifying and representing variability in class models than the Y
approach (SMarty)
 This result corroborates to reject the null hypothesis (H0)
of this study: PLUS is more effective than SMarty for
SPLs class models and subjects taken into account.

21. 21
The Experimental Study
 Threats to Validity
 Conclusion Validity
 Sample size is a major concern, which must be increased in
prospective studies
 Random capacity was not applied to subjects selection, thus
generalization of results could not be inferred
 Construct Validity
 Independent variable variability modeling approach was
guaranteed by the pilot project undertaken

22. 22
The Experimental Study
 Threats to Validity
 Internal Validity
 Differences among subjects – variations on skills were reduced by
performing training session and tasks in the same order
 Fatigue effects – on average the experiment took 80 minutes, thus
fatigue was not considered a problem for the study
 Influence among subjects – could not be really controlled. They took
the experiment under the supervision of a human observer.

23. 23
The Experimental Study
 Threats to Validity
 External Validity
 Instrumentation – failing to use real class models, as the e-commerce
and AGM are not commercial SPLs. More experiments must be
carried out with real SPLs.
 Subjects – we could take advantages of the benefits of taking into
account students for performing experiments as pointed out by
Carver et al. (2003)

24. 24
Empirical Evolution of SMarty
 Improvements for the SMarty approach based on the
feedback of the subjects:
 E-commerce SPL
 Subjects reported difficulties on the application of SMarty to the e-
commerce SPL class model as it has no elements of class modeling,
such as, inheritance, aggregation, and generalization  E-
commerce was analyzed to improve SMarty
 Training session
 Subjects indicated that they need more time for training and
application of SMarty

25. 25
Empirical Evolution of SMarty
 Improvements for the SMarty approach based on the
feedback of the subjects:
 Amount of stereotypes
 Subjects questioned the difference between PLUS and SMarty as
the later has several more stereotypes
 Arrangement of the SPL models:
 E-commerce class models are mode complex than AGM
 E-commerce models were the first models given to the subjects

26. 26
Empirical Evolution of SMarty
 Based on such subjects feedback, the following
improvements were made:
 New guidelines were added to SMarty, encompassing the level of
abstraction and elements of the e-commerce class models
 CL2. Class models elements, related with associations in which their
attributes have aggregationKind as none, or, do not represent
aggregation or composition, suggest mandatory or optional variants.
 CL2.1. For the identification of possible optional variants classes
related through associations where the multiplicity in one of the ends
of an association, to each object class found at the opposite end
match * (zero or more) or 0..n where n is any integer different from
zero; suggest an optional class

27. 27
Empirical Evolution of SMarty
 Based on such subjects feedback, the following
improvements were made:
 We realized that with a few less stereotypes, PLUS is easier to
apply. However, such an “easibility” might jeopardize the
generation of specific products as we can observe ambiguity in
PLUS class models

28. 28
Revisiting objectives of this work…
1. Comparing the effectiveness of SMarty and PLUS with
regard to the identification and representation of
variability in UML classes
 PLUS is more effective
1. Allowing the SMarty evolution in order to increase its
effectiveness against PLUS
 New guidelines for class models  need a new experiment
 Reduce stereotypes or avoid ambiguity during specific products
generation?  New experiment is being carried out

29. 29
Conclusion
 Industry needs that the scientific community tests existing and new
technologies, such as SMarty, identifying their effectiveness to make
technology transfer easier and reliable.
 The experimental study presented in this paper demonstrates the
ability to use variability management approaches. Their
effectiveness were analyzed by modeling variability in class models
of two SPLs.
 Empirical based improvements could be made to SMarty in order to
incorporate support for different class model elements by means of
new guidelines

30. 30
Conclusion
 Shapiro-Wilk normality test was applied to both samples
 Mann-Whitney-Wilcoxon test analyzed the effectiveness of PLUS
and SMarty
 Obtained results provide evidence that the PLUS is more effective
than SMarty for modeling variability in UML class models, taking into
account the E-commerce and the AGM SPLs for this study.

31. 31
Future Works
 New experimental studies and replications must be planned and
conducted to make it possible to reduce the threats, increasing the
effectiveness of SMarty towards generalizing the results.
 As new experiments, we are:
 planning a new experiment for characterize the improved version of
SMarty effective against PLUS; and
 planning an experiment to analyze the effectiveness of PLUS and
SMarty and their impact on the generation of specific products.

32. 32
Acknowledgements
The authors would like to thank
CAPES (a Brazilian Funding Agency)
for supporting this work!!!

33. 33
Questions? Contributions?
 If you prefer, contact me via e-mail:
 edson@din.uem.br

34. 34
Tack! ! !
Thank you ! ! !