Presentation at the FMSPLE workshop held at Eindhoven University of Technology, The Netherlands on 3rd April 2016.

1. Introduction
Background
Review method and evaluation
Future research and conclusion
Analysis of Feature Models using Alloy
A survey
Anjali Sree-Kumar, Elena Planas, Robert Clarisó
SOM Research Lab
Open University of Catalonia, Barcelona
ETAPS - 3rd April @ FMSPLE’16 Workshop, Eindhoven
1 / 24

2. Introduction
Background
Review method and evaluation
Future research and conclusion
Agenda
1 Introduction
FMs & SPL
FM analysis &
automated tools
Purpose of this survey
2 Background
Related surveys &
selected papers
Benchmark feature
model
Alloy semantics
FM analysis operations
3 Review method and
evaluation
Threats to validity
Approach used
Findings
4 Future research and
conclusion
2 / 24

3. Introduction
Background
Review method and evaluation
Future research and conclusion
FMs & SPL
FM analysis & automated tools
Purpose of this survey
Definition - SPL
3 / 24

4. Introduction
Background
Review method and evaluation
Future research and conclusion
FMs & SPL
FM analysis & automated tools
Purpose of this survey
Definition - FM
4 / 24

5. Introduction
Background
Review method and evaluation
Future research and conclusion
FMs & SPL
FM analysis & automated tools
Purpose of this survey
Why FM analysis ?
FM analysis helps in :
1 detecting inconsistencies
2 identifying dead features
3 identifying redundancies
4 optimizing FMs ... and many more ...
5 / 24

6. Introduction
Background
Review method and evaluation
Future research and conclusion
FMs & SPL
FM analysis & automated tools
Purpose of this survey
FM analysis tool - Why Alloy ?
There are no standard text representations for FM.
FM formal specifications must be manually created from
textual requirements.
Alloy - lightweight modeling notation with model finder.
FM specification can be in Alloy notation.
6 / 24

7. Introduction
Background
Review method and evaluation
Future research and conclusion
FMs & SPL
FM analysis & automated tools
Purpose of this survey
Purpose of this survey
Evaluate the State-of-the-art by reviewing 9 relevant
papers.
Characterize the strengths and weaknesses of various
Alloy encoding approaches from the following perspectives
for all the 9 papers :
1 Ease of reproducing the research results.
2 Expressiveness of the FM.
3 Efficiency of the analysis.
4 Existence of tool support.
7 / 24

8. Introduction
Background
Review method and evaluation
Future research and conclusion
Related surveys & selected papers
Benchmark feature model
Alloy semantics
FM analysis operations
Existing surveys in literature
Published research papers with similar survey information
2010 Benavides et al. - Automated analysis of feature models 20
years later : A literature review
2014 Thüm et al. - A Classification and Survey of Analysis
Strategies for Software Product Lines
2015 Lesta et al. - Detecting and Explaining Conflicts in
Attributed Feature Models
8 / 24

9. Introduction
Background
Review method and evaluation
Future research and conclusion
Related surveys & selected papers
Benchmark feature model
Alloy semantics
FM analysis operations
Papers selected for the review
[2005] Wang et al. : Formal Semantics and Verification for Feature
Modeling
[2006] Gheyi et al. : A Theory for Feature Models in Alloy
[2008] Tanizaki et al. : Formalization and Consistency Checking of
Changes of Software System Configurations Using Alloy
[2010] Nakajima et al. : Semi-automated Diagnosis of FODA Feature
Diagram
[2011] Finkel et al. : Reasoning about conditional constraint
specification problems and feature models
[2012] Ripon et al. : Modeling and Analysis of Product line Variants
[2013] Huang et al. : Using Alloy to support feature based DSL
construction for mining software repositories
[2013] Jamie et al. : Supporting Multilevel Configuration with Feature
Solution Graphs Formal Semantics and Alloy
[2015] Ajoudanian et al. : Automatic promotional specialization,
generalization and analysis of extended feature models with
cardinalities in Alloy Implementation
9 / 24

10. Introduction
Background
Review method and evaluation
Future research and conclusion
Related surveys & selected papers
Benchmark feature model
Alloy semantics
FM analysis operations
Benchmark : bCMS (reconstructed in FeatureIDE tool)
This is a case study originating in the http://cserg0.site.uottawa.ca/cma2013models/ "Comparing
Modeling Approaches" at MODELS 2013, which has been used in several other papers as a benchmark FM.
10 / 24

11. Introduction
Background
Review method and evaluation
Future research and conclusion
Related surveys & selected papers
Benchmark feature model
Alloy semantics
FM analysis operations
Alloy example
abstract sig Workshop { }
sig Person { }
one sig FMSPLE extends Workshop
{ audience: set Person }
fact { some FMSPLE.audience };
11 / 24

12. Introduction
Background
Review method and evaluation
Future research and conclusion
Related surveys & selected papers
Benchmark feature model
Alloy semantics
FM analysis operations
Alloy semantics
Constructs Description
Signature sig A set of objects of the same type.
Eg. sig Feature
sig Feature1 extends Feature
Function fun A named expression with 0 or more argu-
ments and a declaration expression for the
result.
Eg. fun satisfyNot[form:Form,
c:Configuration]: Bool {
Not[form.f.satisfy[c]] }
12 / 24

13. Introduction
Background
Review method and evaluation
Future research and conclusion
Related surveys & selected papers
Benchmark feature model
Alloy semantics
FM analysis operations
Alloy semantics (cont.)
Constructs Description
Fact fact A constraint that is assumed to hold in
any instance of the model, e.g. a well-
formedness rule of the model.
Eg. fact { no f: Feature|
f in f.^ parent }
Predicate
pred
A named constraint with 0 or more argu-
ments.
Eg. pred findInstance[ ] {
some featureModel }
run findInstance for 25
13 / 24

14. Introduction
Background
Review method and evaluation
Future research and conclusion
Related surveys & selected papers
Benchmark feature model
Alloy semantics
FM analysis operations
Alloy semantics (cont.)
Constructs Description
Assertion
assert
A constraint that should follow from the
facts of the model.
Eg. assert conflictsShow{ one
fm:FM | fm = featureModel
and (no f:Feature | f in
conflicts[featureModel])}
check conflictsShow
14 / 24

15. Introduction
Background
Review method and evaluation
Future research and conclusion
Related surveys & selected papers
Benchmark feature model
Alloy semantics
FM analysis operations
Operations
15 / 24

16. Introduction
Background
Review method and evaluation
Future research and conclusion
Threats to validity
Approach used
Findings
Threats to validity
Coverage of literature databases :
excluded DBLP and Google Scholar.
included recent related surveys and the references to and
from those papers.
Search strings used :
search based on abstract and keywords.
excluded titles and some special keywords like ’automated
FM analysis’.
Limitations of the case study FM :
not comparable to realistic large scale FMs from actual
industry.
cannot quantify the scalability and ease-of-use of different
Alloy encoding for such realistic FMs.
16 / 24

17. Introduction
Background
Review method and evaluation
Future research and conclusion
Threats to validity
Approach used
Findings
Review process
17 / 24

18. Introduction
Background
Review method and evaluation
Future research and conclusion
Threats to validity
Approach used
Findings
Findings - Encoding families
Two major categories of encoding :
1 Propositional logic based encoding : Relationships are
encoded as formulas
2 Encoding based on Alloy constructs : Relationships are
encoded in Alloy constructs
18 / 24

19. Introduction
Background
Review method and evaluation
Future research and conclusion
Threats to validity
Approach used
Findings
Findings - Pros & Cons
Ability to reason about number of valid product instances
Size or the total number of lines in the Alloy specifications
for large FMs : this varies depending on the encoding
category
Readability of the Alloy specifications/encoding
Degree of expressiveness
19 / 24

20. Introduction
Background
Review method and evaluation
Future research and conclusion
Threats to validity
Approach used
Findings
Findings - Execution time
We were unable to reproduce the results of Ajoudanian
because of the following roadblocks :
Syntax errors in the encoding
Invalid Alloy symbols
Undefined variables
20 / 24

21. Introduction
Background
Review method and evaluation
Future research and conclusion
Threats to validity
Approach used
Findings
Findings - Limitations
Theoretical limitations :
Only one paper supported the counting of valid
configurations.
All encoding supports only trivial relationships.
Practical limitations :
No tool support for generating Alloy specifications for large
FMs. Had to be done manually.
Missing Alloy specifications in some of the reviewed
papers.
No research results validated against very large realistic
FMs.
21 / 24

22. Introduction
Background
Review method and evaluation
Future research and conclusion
Threats to validity
Approach used
Findings
Findings - Summary
Among 30 known FM analysis operations, only 10 are
supported across the 9 research works.
There are no automated tools for translating natural
language specifications or CNF specifications to Alloy
encoding.
Unable to validate the results provided in some research
papers * for our case study on bCMS FM.
Manual encodings one order of magnitude faster than
automatic encodings
Few families of closely related encodings
22 / 24

23. Introduction
Background
Review method and evaluation
Future research and conclusion
Future research and conclusion
Research on the possibility of Alloy being able to support
all the remaining analysis operations.
Emphasized the importance of automated tool support for
converting FM diagrams to Alloy notations.
Identified the limitations of the available research results
and its application in our case study.
Highlighted the importance of future research on
automated FM analysis using Alloy.
23 / 24

24. Introduction
Background
Review method and evaluation
Future research and conclusion
Thank you
Thank you for your attention !
Contact : Anjali Sree-Kumar
Email : asree_kumar@uoc.edu
Case study Alloy source files URL :
https://www.dropbox.com/s/wysbls64ictoofm/Alloy.zip?dl=0
24 / 24