Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Scalable Product Line Configuration:
A Straw to Break the Camel’s Back
Abdel Salam Sayyad
Joseph Ingram
Tim Menzies
Hany A...
Sound bites
Real Software Product Lines…
… are big, complex
… representative feature models are now available for research...
Roadmap

① State of the Art
② Experiment & Results
③ Conclusion
Roadmap

① State of the Art
② Experiment & Results
③ Conclusion
Feature Models
• Feature models = a
lightweight method for
defining a space of options
• De facto standard for
modeling va...
Feature Model Repository

Size ? 290 Features, 421 Rules
cardinal = 2.26x1049 *Pohl ‘11+
6
Feature Models of Real Software Projects

Size ? 6888 Features; 344,000 Rules
cardinal? Forgeddaboutit!

7
Size isn’t all…
Property
Size

SPLOT
Significantly smaller

Linux
Significantly larger

Constraints
Feature Groups
Leaves
...
A Multi-Objective Formulation
Multi-objective optimization = navigating competing concerns
– Success criteria = choose fea...
No single “optimum” solution

The Pareto Front

Higher-level
Decision Making

The Chosen Solution

10
State of the Art

Features

Linux (LVAT)

6888

544

Single-objective

Johansen
‘11

Henard
‘12

Multi-objective

Sayyad ’...
Roadmap

① State of the Art
② Experiment & Results
③ Conclusion
Two Algorithms
1) NSGA-II [Deb et al. ‘02+
Non-dominated Sorting
Genetic Algorithm
Binary Dominance
2) IBEA *Zitzler and K...
7 Feature Models

14
Feature Fixing
• Look for mandatory or dead features.
• Fix those feature in the evolution.
• Skip rules with only those f...
Experiment

Low Crossover & Mutation rates
Stopping Criteria

16
HV
= Hypervolume of dominated region
%Correct = % of solutions that are fully correct (out of 300 solutions)

17
Comparing %Correct
i.e. % of solutions that are fully correct (out of 300 solutions)
NSGA-II No FF

100%

NSGA-II with FF
...
How to crack the nut?
• What if we told IBEA what good solutions look
like?
• Good solution? … A correct solution, with:
•...
• How to generate good (correct) solutions?
• Z3 SMT (Satisfiability Module Theory) Solver
• Two-objective IBEA, min viola...
Correct solutions after 30 minutes for
Linux Kernel feature model

21
Correct solutions after 30 minutes for
Linux Kernel feature model
… A Straw to Break the Camel’s Back

22
More recent result
• added the “PULL” trick (give more objective weight
to constraint violations)

23
Why is this interesting?
• 20 correct solutions to choose from within 2 minutes

24
Do I have the user’s attention?
• 2 seconds…
• to stay focused

• 10 seconds…
• to stay on task

• few minutes?
• To provi...
Roadmap

① State of the Art
② Experiment & Results
③ Conclusion
Sound bites
Scalability
Method innovation is key

Problem Formulation
Give the best (comprehensive yet
concise) picture to...
Upcoming SlideShare
Loading in …5
×

Scalable Product Line Configuration - ASE 2013 Palo Alto, CA

726 views

Published on

My Talk at ASE conference

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Scalable Product Line Configuration - ASE 2013 Palo Alto, CA

  1. 1. Scalable Product Line Configuration: A Straw to Break the Camel’s Back Abdel Salam Sayyad Joseph Ingram Tim Menzies Hany Ammar
  2. 2. Sound bites Real Software Product Lines… … are big, complex … representative feature models are now available for research … scale up or die ! What does it take? Multi-objective problem formulation An optimizer that relies heavily on user preferences Respect domain constraints, and... … A Straw to Break the Camel’s Back 2
  3. 3. Roadmap ① State of the Art ② Experiment & Results ③ Conclusion
  4. 4. Roadmap ① State of the Art ② Experiment & Results ③ Conclusion
  5. 5. Feature Models • Feature models = a lightweight method for defining a space of options • De facto standard for modeling variability Cross-Tree Constraints Size ? 10 Features, 8 Rules Cross-Tree Constraints 5
  6. 6. Feature Model Repository Size ? 290 Features, 421 Rules cardinal = 2.26x1049 *Pohl ‘11+ 6
  7. 7. Feature Models of Real Software Projects Size ? 6888 Features; 344,000 Rules cardinal? Forgeddaboutit! 7
  8. 8. Size isn’t all… Property Size SPLOT Significantly smaller Linux Significantly larger Constraints Feature Groups Leaves Significantly less High ratio Deeper Significantly more Low ratio Shallower T. Berger, S. She, R. Lotufo, A. Wasowski, and K. Czarnecki, "A Study of Variability Models and Languages in the Systems Software Domain," IEEE Tran Soft Eng. 2013 8
  9. 9. A Multi-Objective Formulation Multi-objective optimization = navigating competing concerns – Success criteria = choose features that achieve the user preferences! Suppose each feature had the following metrics: 1. Boolean 2. Integer 3. Real USED_BEFORE? DEFECTS COST Show me the space of “best options” according to the objectives: 1. 2. 3. 4. 5. That satisfies most domain constraints (0 ≤ #violations ≤ 100%) That offers most features That we have used most before Using features with least known defects Using features with least cost 9
  10. 10. No single “optimum” solution The Pareto Front Higher-level Decision Making The Chosen Solution 10
  11. 11. State of the Art Features Linux (LVAT) 6888 544 Single-objective Johansen ‘11 Henard ‘12 Multi-objective Sayyad ’13b White ‘07, ‘08, 09a, 09b, Shi ‘10, Guo ‘11 SPLOT 290 9 Pohl ‘11 Benavides ‘05 LopezHerrejon ‘11 Sayyad ’13a Olaechea ‘13 Objectives 11
  12. 12. Roadmap ① State of the Art ② Experiment & Results ③ Conclusion
  13. 13. Two Algorithms 1) NSGA-II [Deb et al. ‘02+ Non-dominated Sorting Genetic Algorithm Binary Dominance 2) IBEA *Zitzler and Kunzli ‘04] Indicator-Based Evolutionary Algorithm Continuous Dominance 13
  14. 14. 7 Feature Models 14
  15. 15. Feature Fixing • Look for mandatory or dead features. • Fix those feature in the evolution. • Skip rules with only those features. 15
  16. 16. Experiment Low Crossover & Mutation rates Stopping Criteria 16
  17. 17. HV = Hypervolume of dominated region %Correct = % of solutions that are fully correct (out of 300 solutions) 17
  18. 18. Comparing %Correct i.e. % of solutions that are fully correct (out of 300 solutions) NSGA-II No FF 100% NSGA-II with FF 80% IBEA No FF IBEA with FF 60% 40% 20% 0% ToyBox axTLS eCos FreeBSD Fiasco uClinux Linux X86 18
  19. 19. How to crack the nut? • What if we told IBEA what good solutions look like? • Good solution? … A correct solution, with: • Minimal skeleton? • Maximum features? • Then what? • Seed the initial population with a pre-calculated “good” solution. 19
  20. 20. • How to generate good (correct) solutions? • Z3 SMT (Satisfiability Module Theory) Solver • Two-objective IBEA, min violations, max features • Z3: super fast, low on selected features • 2-obj IBEA: slow, but feature-rich 20
  21. 21. Correct solutions after 30 minutes for Linux Kernel feature model 21
  22. 22. Correct solutions after 30 minutes for Linux Kernel feature model … A Straw to Break the Camel’s Back 22
  23. 23. More recent result • added the “PULL” trick (give more objective weight to constraint violations) 23
  24. 24. Why is this interesting? • 20 correct solutions to choose from within 2 minutes 24
  25. 25. Do I have the user’s attention? • 2 seconds… • to stay focused • 10 seconds… • to stay on task • few minutes? • To provide timely input at a meeting 25
  26. 26. Roadmap ① State of the Art ② Experiment & Results ③ Conclusion
  27. 27. Sound bites Scalability Method innovation is key Problem Formulation Give the best (comprehensive yet concise) picture to the user. IBEA with Feature Fixing Continuous dominance Respect domain constraints Acknowledgment This research work was funded by the Qatar National Research Fund (QNRF) under the National Priorities Research Program (NPRP) Grant No.: 09-1205-2-470. Enter Z3? One day we will build that bridge! 27

×