Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
Francis Palma
Department of Information Engineering and Computer Science
UniversitĂĄ degli Studi di Trento
francis.palma@studenti.unitn.it
December 3, 2010
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Presentation Outline
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
 Problem description
 Related works
 Genetic Algorithm
 Our approach
 Case study
 Results
 Conclusions
2
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Problem Description
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
Problem: [Prioritization of Requirements] To find the best ordering of
requirements in each successive release to ensure quality & value of
the delivered system, trade-off constraints & end-user satisfaction. 3
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Do Trade-off between user needs & real constraints
Now, highest quality &
best valued system!
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
4
1. Acquisition and coding of set of Requirements and Domain Knowledge
2. Apply A Prioritization Technique
3. Output of the ranking
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Problem Description (cont)
Find it!Why Ordered List? Order of implementation satisfy the
developers’ constraints & delivers maximum value to the user.
Domain Knowledge: Prio & Dep
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
5
Domain knowledge includes two precedence graphs
• Prio
• Dep
How to build them?
Prio: For 1st release, initial priorities
Dep: For 1st release, dependencies
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Prio
Dep
Classification: State-of-the-Art approaches
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
6
Prioritization Approaches Classification
using user knowledge either performing pair-wise
comparison or not
using domain knowledge
using both
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
 User Knowledge refers to the awareness of the requirements attributes
and the overall system functionalities to be developed
State-of-the-Art approaches (cont)
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
7
Pairwise Comparison based approaches
Analytic Hierarchy Process (AHP): involves comparing all unique pairs of
requirements to determine which of the two is of higher priority, and to
what extent
Bubble Sort: compares two requirements & swap them if they are in the
wrong order
Cost-Value Approach: each individual requirement is determined on
(i) the value to the users
(ii) the cost of implementing the requirements.
It uses the AHP technique
Case Based Ranking (CBRank): exploits a machine learning algorithm to
guide the elicitation of user preferences during the prioritization process
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
State-of-the-Art approaches (cont)
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
8
Non-Pairwise comparison based approaches
Numerical Assignment: grouping requirements into different priority groups
MoScoW: groups all requirements into four priority groups MUST have, SHOULD have,
COULD have, and WON’T have.
Simple Ranking: requirements are simply ranked from integer 1 to N
Binary Search Tree: each node represents a requirement, requirements placed in the
left subtree of a node are of lower priority and those placed in the right subtree of a
node are of higher priority than the node priority.
$100 Method: each stakeholder is asked to assume having $100 to distribute over the
requirements in a ratio scale
Combining Techniques based approaches
Planning Game: combination of two prioritization techniques i.e. Numerical Assignment
& Simple Ranking
Domain Knowledge based approaches
Priority Groups: dividing requirements into separate groups. then groups are ranked by
using AHP
Genetic Algorithm: optimization is an application of GA & used in the problem of
requirements prioritization too; uses domain knowledge
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
State-of-the-Art approaches (cont)
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
9
Summary
Approach Cons
AHP Scalability
Bubble Sort Scalability, time
CBRank unable to accept constraints i.e. Dep
Cost-Value Approach Time consuming
BST Sensitivity; a single error may build wrong tree
GA Can’t resolve contradictory;
$100 Method Longer; less confidence, biased
MoScoW Ambiguous final ordering
Simple Ranking unable handling complex scenarios
Scalability is a very common problem of all the approaches!
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Genetic Algorithm
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
10
1. Acquisition and coding of set of Requirements and Domain Knowledge
2. Apply Genetic Algorithm (no user knowledge)
3. Output of the ranking (the most promising individual)
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Genetic Algorithm Pseudo Code
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
11
The Canonical GA (a simple pseudo code is presented here):
1. choose initial population
2. evaluate each individual’s fitness
REPEAT:
3. select best-ranking individuals to reproduce
4. apply crossover operator
5. apply mutation operator
6. evaluate each individual’s fitness
until terminating condition (e.g. until at least one individual
has the desired fitness or enough generations have passed)
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
What is Population & Individuals?
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
12
Indv. ID Individual Disagree
Pr1 < R1, R3, R2, R4, R5, R6, R7, R8, R9 >
Pr2 < R2, R3, R4, R1, R5, R8, R6, R7, R9 >
Pr3 < R5, R2, R1, R3, R7, R8, R6, R9, R4 >
Pr4 < R4, R5, R6, R3, R2, R1, R8, R9, R7 >
Pr5 < R7, R8, R6, R5, R2, R3, R4, R9, R1 >
Pr6 < R5, R6, R7, R8, R9, R1, R2, R3, R4 >
Pr7 < R9, R8, R7, R6, R5, R4, R3, R2, R1 >
Pr8 < R8, R9, R6, R7, R4, R5, R2, R3, R1 >
Pr9 < R1, R3, R5, R7, R9, R2, R4, R6, R8 >
Pr10 < R1, R4, R2, R3, R5, R6, R9, R8, R7 >
R1 R3 R2 R4 R5 R6 R7 R8 R9
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Population!
A set of
solution
candidates
Pr1
An individual, also a solution candidate
A requirement as a gene
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
13
How we define Individual’s Fitness?
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Domain Knowledge Encoding
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
14
Transform the domain knowledge into graphs
Req Priorities Dependencies
R1 10 R2, R3, R7
R2 20 R3
R3 10
R4 20
R5 30 R4, R8
R6 40
R7 30 R3
R8 20
R9 50 R8, R5
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Disagreement Calculation
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
15
Indv. ID Individual Disagree
Pr1 < R1, R3, R2, R4, R5, R6, R7, R8, R9 >
Pr2 < R2, R3, R4, R1, R5, R8, R6, R7, R9 >
Pr3 < R5, R2, R1, R3, R7, R8, R6, R9, R4 >
Pr4 < R4, R5, R6, R3, R2, R1, R8, R9, R7 >
Pr5 < R7, R8, R6, R5, R2, R3, R4, R9, R1 >
Pr6 < R5, R6, R7, R8, R9, R1, R2, R3, R4 >
Pr7 < R9, R8, R7, R6, R5, R4, R3, R2, R1 >
Pr8 < R8, R9, R6, R7, R4, R5, R2, R3, R1 >
Pr9 < R1, R3, R5, R7, R9, R2, R4, R6, R8 >
Pr10 < R1, R4, R2, R3, R5, R6, R9, R8, R7 >
R1 R3 R2 R4 R5 R6 R7 R8 R9
Conflicts =
{(R5, R8),
(R6, R7),
(R6, R8),
(R7, R8)}
Prio
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
For Pr1
Disagreement Calculation (cont)
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
16
Indv. ID Individuals Disagree
Pr1 < R1, R3, R2, R4, R5, R6, R7, R8, R9 >
Pr2 < R2, R3, R4, R1, R5, R8, R6, R7, R9 >
Pr3 < R5, R2, R1, R3, R7, R8, R6, R9, R4 >
Pr4 < R4, R5, R6, R3, R2, R1, R8, R9, R7 >
Pr5 < R7, R8, R6, R5, R2, R3, R4, R9, R1 >
Pr6 < R5, R6, R7, R8, R9, R1, R2, R3, R4 >
Pr7 < R9, R8, R7, R6, R5, R4, R3, R2, R1 >
Pr8 < R8, R9, R6, R7, R4, R5, R2, R3, R1 >
Pr9 < R1, R3, R5, R7, R9, R2, R4, R6, R8 >
Pr10 < R1, R4, R2, R3, R5, R6, R9, R8, R7 >
R1 R3 R2 R4 R5 R6 R7 R8 R9
Conflicts =
{(R1, R3),
(R1, R7),
(R1, R2),
(R5, R8)}
Dep
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
For Pr1
Disagreement Calculation (cont)
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
17
Indv. ID Individuals Disagree
Pr1 < R1, R3, R2, R4, R5, R6, R7, R8, R9 > 8
Pr2 < R2, R3, R4, R1, R5, R8, R6, R7, R9 > 8
Pr3 < R5, R2, R1, R3, R7, R8, R6, R9, R4 > 16
Pr4 < R4, R5, R6, R3, R2, R1, R8, R9, R7 > 15
Pr5 < R7, R8, R6, R5, R2, R3, R4, R9, R1 > 23
Pr6 < R5, R6, R7, R8, R9, R1, R2, R3, R4 > 29
Pr7 < R9, R8, R7, R6, R5, R4, R3, R2, R1 > 30
Pr8 < R8, R9, R6, R7, R4, R5, R2, R3, R1 > 29
Pr9 < R1, R3, R5, R7, R9, R2, R4, R6, R8 > 17
Pr10 < R1, R4, R2, R3, R5, R6, R9, R8, R7 > 13
Total Conflicts =
{(R5, R8), Prio
(R6, R7),
(R6, R8),
(R7, R8),
(R1, R3), Dep
(R1, R7),
(R1, R2),
(R5, R8)}
… and so on …
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions

dis(pr1, pr2) {(r,s)  pr1* |(r,s)  pr2*}
 Disagreement: Between a pair of ordering i.e. Pr1
and Prio, disagreement is the count of pairs that are inverted in
two orderings. Lower disagreement defines higher fitness.
Formally define:
GA Crossover Operator
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
18
We used cut-head/fill-in-tail and cut-tail/fill-in-head…
R2 R3
Positions 5-6 as cut points
to cut-head/fill-in-tail
R6
Pr2
Pr3
Pr2’
R4 R1 R5 R8 R6 R7 R9
R5 R8 R1 R3 R7 R2 R9 R4 R6
R7 R8 R9R2 R3 R4 R1 R5
- variation allows searching out different available niches, find better
fitness values and subsequently better solutions
- never produce chromosomes containing duplicate genes.
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
GA Mutation Operator
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
19
Mutation is basically a swap operator, we used
requirement-pair-swap
R2 R3
Pr2
R4 R1 R5 R8 R6 R7 R9
Pr2’
R2 R3 R6 R1 R5 R8 R4 R7 R9
- mutation causes movement in the search space
- may produce a stronger chromosome.
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
GA Selection Operator
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
20
We used the Tournament Selection for our approach
- allows the selection pressure to be easily adjusted
- faster than other selection operators i.e. Roulette Wheel Selection
- better convergence
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Our IGA Approach
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
21
1. Acquisition and coding of set of Requirements and Domain Knowledge
2. Apply Interactive Genetic Algorithm (exploiting User Knowledge)
3. Output of the ranking (the most promising individual)
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Our Approach: Pairs to be Elicited
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
22
Indv. ID Individuals Disagree
Pr1 < R1, R3, R2, R4, R5, R6, R7, R8, R9 > 8
Pr2 < R2, R3, R4, R1, R5, R8, R6, R7, R9 > 8
Pr10 < R1, R4, R2, R3, R5, R6, R9, R8, R7 > 13
Pr4 < R4, R5, R6, R3, R2, R1, R8, R9, R7 > 15
Pr3 < R5, R2, R1, R3, R7, R8, R6, R9, R4 > 16
Pr9 < R1, R3, R5, R7, R9, R2, R4, R6, R8 > 17
Pr5 < R7, R8, R6, R5, R2, R3, R4, R9, R1 > 23
Pr8 < R8, R9, R6, R7, R4, R5, R2, R3, R1 > 29
Pr6 < R5, R6, R7, R8, R9, R1, R2, R3, R4 > 29
Pr7 < R9, R8, R7, R6, R5, R4, R3, R2, R1 > 30
Ranked
individuals
w. r. t.
disagreement
Pr1 = < R1, R3, R2, R4, R5, R6, R7, R8, R9 >
vs
Pr2 = < R2, R3, R4, R1, R5, R8, R6, R7, R9 >
TIE PAIRS
Pr1, Pr2 (R1, R3), (R2, R3), (R6, R8), (R7, R8)
Candidate pairs to be
asked to the decision
maker
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Our Approach: User Feedback
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
23
Why ties appear in population?
(i) Contradictory information
w.r.t. initial constraints
(ii) Nothing is expressed
explicitly in initial constraints
(iii) Common Domain
Knowledge but different
positions in the individuals
Simple example:
Why (R7, R8)?
Case I: Contradictory w.r.t. Prio & Dep..
Why (R2, R3)?
Case III: Common knowledge but different positions
TIE PAIRS
Pr1, Pr2 (R1, R3), (R2, R3), (R6, R8), (R7, R8) Prio
Dep
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Our Approach: User Feedback
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
24
TIE PAIRS
Pr1, Pr2 (R1, R3), (R2, R3), (R6, R8), (R7, R8)
Eli
User Preference Graph eliOrd
Experience
&
knowledge
‘<‘ or ‘>’
So, user knowledge is playing important role…
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Our Approach: New Round with New Constraints
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
25
 The new evolved population after using GA operators
on population:
 Crossover
 Mutation
 Selection
 is compared against the new set of constraints graphs
Eli
DepPrio
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
26
Evaluation of the Approach
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
The Case Study
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
27
 Prioritize requirements for a real software system, as part of the
project ACube (Ambient Aware Assistance)
designing a highly technological monitoring environment to be
deployed in nursing homes to support medical and assistance staff
 After user requirements analysis phase,
60 user requirements and 49 technical requirements
Four macro-scenarios have been identified.
ID Macro-Scenario # of requirements
FALL Monitoring falls 26
ESC Monitoring escapes 23
MON Monitoring dangerous behavior 21
ALL The three scenarios 49
*ACube is a social welfare project coordinated by Fondazione Bruno Kessler (FBK) and
funded by Autonomous Province of Trento under Bando Grandi Progetti, 2006.
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Gold Standard (GS)
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
28
 For each of the four macro-scenarios, we obtained the
Gold Standard (GS) prioritization from the Software Architect
of the ACube project
 The GS prioritization is the ordering given by the software
architect to the requirements when planning their
implementation during the ACube project.
Why Gold Standard?
 To measure disagreements with respect to GS.
 To evaluate our approach in terms of disagreement
against other non-interactive approaches using the same
GS.
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
29
Research Questions
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
R Q 1
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
30
RQ1 (Convergence) Can we observe convergence with respect to the
finally elicited fitness function?
- Convergence is not obvious immediately, as Eli Graph is evolving at early stages.
- Although the full fitness function is known only at end of elicitation process.
The best individual in each population converges toward a low value of the
final fitness function.Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
YES!
R Q 2
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
31
RQ2 (Role of interaction) Does IGA produce improved
prioritizations compared to non-interactive requirement ordering?
IGA outperforms substantially GA (and RAND), especially when a
higher number of pairwise comparisons can be carried out
YES!
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
R Q 3
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
32
RQ3 (Role of initial precedence constraints) How does initial
availability of precedence constraints affect the performance of IGA?
- Different type of Domain Knowledge affects IGA significantly
- The improvement of IGA over GA is even higher when limited ranking
information is available
Improves!
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
R Q 4
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
33
RQ4 (Robustness) Is IGA robust with respect to errors committed by
the user during the elicitation of pairwise comparisons?
IGA can tolerate user errors up to 20%.
YES!Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
General Discussion
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
34
 Cost/benefit trade off offered by IGA as compared to
AHP is extremely interesting
 With an elicitation effort reduced to 10% of the one
required by AHP, IGA produces an apprx. ordering which has
a quite low disagreement from the requirement positions in
the GS.
 User Errors tolerance offered by IGA as compared to
AHP is highly reasonable
 With an elicitation of very significant amount of less pairs
than AHP (i.e. pairwise comparisons), it is reasonable even if
IGA accepts up to 20% user error, while using AHP even 10%
user error can result a total bad ordering.
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Conclusions & Future Works
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
35
 We proposed an Interactive Genetic Algorithm to collect
pairwise information useful to prioritize the requirements for a
software system.
 We also verified the robustness of the algorithm with respect
to increasing user feedback errors.
 We evaluated the approach in a real project (ACube).
 In summary, we contributed A NOVEL APPROACH to prioritize
requirements & tested its effectiveness empirically.
What’s Next?
Algorithm:
- refining the algorithm
- improving GA operators
Experiment
- off-line: comparisons with other approaches
- on-line: controlled experiments with real object (i.e. human)
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Paper Publication
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
36
A paper was published from this work and was presented at SSBSE
2010:
Paolo Tonella1, Angelo Susi1, and Francis Palma2, "Using
Interactive GA for Requirements Prioritization" in 2nd
International Symposium on Search Based Software Engineering
2010. 1Fondazione Bruno Kessler, Software Engineering Research
Unit; 2Department of Inf. Eng. and Computer Science, University of
Trento.
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions
Using
Interactive
Genetic
Algorithm for
Requirements
Prioritization
37
Thank You!
Outline
Problem
Related Works
Genetic Algo
Our Approach
Case Study
Results
Discussions
Conclusions

Using Interactive Genetic Algorithm for Requirements Prioritization

  • 1.
    Using Interactive Genetic Algorithm for Requirements Prioritization Francis Palma Departmentof Information Engineering and Computer Science UniversitĂĄ degli Studi di Trento francis.palma@studenti.unitn.it December 3, 2010 Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 2.
    Presentation Outline Using Interactive Genetic Algorithm for Requirements Prioritization Problem description  Related works  Genetic Algorithm  Our approach  Case study  Results  Conclusions 2 Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 3.
    Problem Description Using Interactive Genetic Algorithm for Requirements Prioritization Problem:[Prioritization of Requirements] To find the best ordering of requirements in each successive release to ensure quality & value of the delivered system, trade-off constraints & end-user satisfaction. 3 Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions Do Trade-off between user needs & real constraints Now, highest quality & best valued system!
  • 4.
    Using Interactive Genetic Algorithm for Requirements Prioritization 4 1. Acquisitionand coding of set of Requirements and Domain Knowledge 2. Apply A Prioritization Technique 3. Output of the ranking Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions Problem Description (cont) Find it!Why Ordered List? Order of implementation satisfy the developers’ constraints & delivers maximum value to the user.
  • 5.
    Domain Knowledge: Prio& Dep Using Interactive Genetic Algorithm for Requirements Prioritization 5 Domain knowledge includes two precedence graphs • Prio • Dep How to build them? Prio: For 1st release, initial priorities Dep: For 1st release, dependencies Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions Prio Dep
  • 6.
    Classification: State-of-the-Art approaches Using Interactive Genetic Algorithmfor Requirements Prioritization 6 Prioritization Approaches Classification using user knowledge either performing pair-wise comparison or not using domain knowledge using both Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions  User Knowledge refers to the awareness of the requirements attributes and the overall system functionalities to be developed
  • 7.
    State-of-the-Art approaches (cont) Using Interactive Genetic Algorithmfor Requirements Prioritization 7 Pairwise Comparison based approaches Analytic Hierarchy Process (AHP): involves comparing all unique pairs of requirements to determine which of the two is of higher priority, and to what extent Bubble Sort: compares two requirements & swap them if they are in the wrong order Cost-Value Approach: each individual requirement is determined on (i) the value to the users (ii) the cost of implementing the requirements. It uses the AHP technique Case Based Ranking (CBRank): exploits a machine learning algorithm to guide the elicitation of user preferences during the prioritization process Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 8.
    State-of-the-Art approaches (cont) Using Interactive Genetic Algorithmfor Requirements Prioritization 8 Non-Pairwise comparison based approaches Numerical Assignment: grouping requirements into different priority groups MoScoW: groups all requirements into four priority groups MUST have, SHOULD have, COULD have, and WON’T have. Simple Ranking: requirements are simply ranked from integer 1 to N Binary Search Tree: each node represents a requirement, requirements placed in the left subtree of a node are of lower priority and those placed in the right subtree of a node are of higher priority than the node priority. $100 Method: each stakeholder is asked to assume having $100 to distribute over the requirements in a ratio scale Combining Techniques based approaches Planning Game: combination of two prioritization techniques i.e. Numerical Assignment & Simple Ranking Domain Knowledge based approaches Priority Groups: dividing requirements into separate groups. then groups are ranked by using AHP Genetic Algorithm: optimization is an application of GA & used in the problem of requirements prioritization too; uses domain knowledge Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 9.
    State-of-the-Art approaches (cont) Using Interactive Genetic Algorithmfor Requirements Prioritization 9 Summary Approach Cons AHP Scalability Bubble Sort Scalability, time CBRank unable to accept constraints i.e. Dep Cost-Value Approach Time consuming BST Sensitivity; a single error may build wrong tree GA Can’t resolve contradictory; $100 Method Longer; less confidence, biased MoScoW Ambiguous final ordering Simple Ranking unable handling complex scenarios Scalability is a very common problem of all the approaches! Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 10.
    Genetic Algorithm Using Interactive Genetic Algorithm for Requirements Prioritization 10 1.Acquisition and coding of set of Requirements and Domain Knowledge 2. Apply Genetic Algorithm (no user knowledge) 3. Output of the ranking (the most promising individual) Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 11.
    Genetic Algorithm PseudoCode Using Interactive Genetic Algorithm for Requirements Prioritization 11 The Canonical GA (a simple pseudo code is presented here): 1. choose initial population 2. evaluate each individual’s fitness REPEAT: 3. select best-ranking individuals to reproduce 4. apply crossover operator 5. apply mutation operator 6. evaluate each individual’s fitness until terminating condition (e.g. until at least one individual has the desired fitness or enough generations have passed) Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 12.
    What is Population& Individuals? Using Interactive Genetic Algorithm for Requirements Prioritization 12 Indv. ID Individual Disagree Pr1 < R1, R3, R2, R4, R5, R6, R7, R8, R9 > Pr2 < R2, R3, R4, R1, R5, R8, R6, R7, R9 > Pr3 < R5, R2, R1, R3, R7, R8, R6, R9, R4 > Pr4 < R4, R5, R6, R3, R2, R1, R8, R9, R7 > Pr5 < R7, R8, R6, R5, R2, R3, R4, R9, R1 > Pr6 < R5, R6, R7, R8, R9, R1, R2, R3, R4 > Pr7 < R9, R8, R7, R6, R5, R4, R3, R2, R1 > Pr8 < R8, R9, R6, R7, R4, R5, R2, R3, R1 > Pr9 < R1, R3, R5, R7, R9, R2, R4, R6, R8 > Pr10 < R1, R4, R2, R3, R5, R6, R9, R8, R7 > R1 R3 R2 R4 R5 R6 R7 R8 R9 Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions Population! A set of solution candidates Pr1 An individual, also a solution candidate A requirement as a gene
  • 13.
    Using Interactive Genetic Algorithm for Requirements Prioritization 13 How wedefine Individual’s Fitness? Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 14.
    Domain Knowledge Encoding Using Interactive Genetic Algorithmfor Requirements Prioritization 14 Transform the domain knowledge into graphs Req Priorities Dependencies R1 10 R2, R3, R7 R2 20 R3 R3 10 R4 20 R5 30 R4, R8 R6 40 R7 30 R3 R8 20 R9 50 R8, R5 Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 15.
    Disagreement Calculation Using Interactive Genetic Algorithm for Requirements Prioritization 15 Indv.ID Individual Disagree Pr1 < R1, R3, R2, R4, R5, R6, R7, R8, R9 > Pr2 < R2, R3, R4, R1, R5, R8, R6, R7, R9 > Pr3 < R5, R2, R1, R3, R7, R8, R6, R9, R4 > Pr4 < R4, R5, R6, R3, R2, R1, R8, R9, R7 > Pr5 < R7, R8, R6, R5, R2, R3, R4, R9, R1 > Pr6 < R5, R6, R7, R8, R9, R1, R2, R3, R4 > Pr7 < R9, R8, R7, R6, R5, R4, R3, R2, R1 > Pr8 < R8, R9, R6, R7, R4, R5, R2, R3, R1 > Pr9 < R1, R3, R5, R7, R9, R2, R4, R6, R8 > Pr10 < R1, R4, R2, R3, R5, R6, R9, R8, R7 > R1 R3 R2 R4 R5 R6 R7 R8 R9 Conflicts = {(R5, R8), (R6, R7), (R6, R8), (R7, R8)} Prio Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions For Pr1
  • 16.
    Disagreement Calculation (cont) Using Interactive Genetic Algorithmfor Requirements Prioritization 16 Indv. ID Individuals Disagree Pr1 < R1, R3, R2, R4, R5, R6, R7, R8, R9 > Pr2 < R2, R3, R4, R1, R5, R8, R6, R7, R9 > Pr3 < R5, R2, R1, R3, R7, R8, R6, R9, R4 > Pr4 < R4, R5, R6, R3, R2, R1, R8, R9, R7 > Pr5 < R7, R8, R6, R5, R2, R3, R4, R9, R1 > Pr6 < R5, R6, R7, R8, R9, R1, R2, R3, R4 > Pr7 < R9, R8, R7, R6, R5, R4, R3, R2, R1 > Pr8 < R8, R9, R6, R7, R4, R5, R2, R3, R1 > Pr9 < R1, R3, R5, R7, R9, R2, R4, R6, R8 > Pr10 < R1, R4, R2, R3, R5, R6, R9, R8, R7 > R1 R3 R2 R4 R5 R6 R7 R8 R9 Conflicts = {(R1, R3), (R1, R7), (R1, R2), (R5, R8)} Dep Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions For Pr1
  • 17.
    Disagreement Calculation (cont) Using Interactive Genetic Algorithmfor Requirements Prioritization 17 Indv. ID Individuals Disagree Pr1 < R1, R3, R2, R4, R5, R6, R7, R8, R9 > 8 Pr2 < R2, R3, R4, R1, R5, R8, R6, R7, R9 > 8 Pr3 < R5, R2, R1, R3, R7, R8, R6, R9, R4 > 16 Pr4 < R4, R5, R6, R3, R2, R1, R8, R9, R7 > 15 Pr5 < R7, R8, R6, R5, R2, R3, R4, R9, R1 > 23 Pr6 < R5, R6, R7, R8, R9, R1, R2, R3, R4 > 29 Pr7 < R9, R8, R7, R6, R5, R4, R3, R2, R1 > 30 Pr8 < R8, R9, R6, R7, R4, R5, R2, R3, R1 > 29 Pr9 < R1, R3, R5, R7, R9, R2, R4, R6, R8 > 17 Pr10 < R1, R4, R2, R3, R5, R6, R9, R8, R7 > 13 Total Conflicts = {(R5, R8), Prio (R6, R7), (R6, R8), (R7, R8), (R1, R3), Dep (R1, R7), (R1, R2), (R5, R8)} … and so on … Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions  dis(pr1, pr2) {(r,s)  pr1* |(r,s)  pr2*}  Disagreement: Between a pair of ordering i.e. Pr1 and Prio, disagreement is the count of pairs that are inverted in two orderings. Lower disagreement defines higher fitness. Formally define:
  • 18.
    GA Crossover Operator Using Interactive Genetic Algorithmfor Requirements Prioritization 18 We used cut-head/fill-in-tail and cut-tail/fill-in-head… R2 R3 Positions 5-6 as cut points to cut-head/fill-in-tail R6 Pr2 Pr3 Pr2’ R4 R1 R5 R8 R6 R7 R9 R5 R8 R1 R3 R7 R2 R9 R4 R6 R7 R8 R9R2 R3 R4 R1 R5 - variation allows searching out different available niches, find better fitness values and subsequently better solutions - never produce chromosomes containing duplicate genes. Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 19.
    GA Mutation Operator Using Interactive Genetic Algorithmfor Requirements Prioritization 19 Mutation is basically a swap operator, we used requirement-pair-swap R2 R3 Pr2 R4 R1 R5 R8 R6 R7 R9 Pr2’ R2 R3 R6 R1 R5 R8 R4 R7 R9 - mutation causes movement in the search space - may produce a stronger chromosome. Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 20.
    GA Selection Operator Using Interactive Genetic Algorithmfor Requirements Prioritization 20 We used the Tournament Selection for our approach - allows the selection pressure to be easily adjusted - faster than other selection operators i.e. Roulette Wheel Selection - better convergence Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 21.
    Our IGA Approach Using Interactive Genetic Algorithmfor Requirements Prioritization 21 1. Acquisition and coding of set of Requirements and Domain Knowledge 2. Apply Interactive Genetic Algorithm (exploiting User Knowledge) 3. Output of the ranking (the most promising individual) Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 22.
    Our Approach: Pairsto be Elicited Using Interactive Genetic Algorithm for Requirements Prioritization 22 Indv. ID Individuals Disagree Pr1 < R1, R3, R2, R4, R5, R6, R7, R8, R9 > 8 Pr2 < R2, R3, R4, R1, R5, R8, R6, R7, R9 > 8 Pr10 < R1, R4, R2, R3, R5, R6, R9, R8, R7 > 13 Pr4 < R4, R5, R6, R3, R2, R1, R8, R9, R7 > 15 Pr3 < R5, R2, R1, R3, R7, R8, R6, R9, R4 > 16 Pr9 < R1, R3, R5, R7, R9, R2, R4, R6, R8 > 17 Pr5 < R7, R8, R6, R5, R2, R3, R4, R9, R1 > 23 Pr8 < R8, R9, R6, R7, R4, R5, R2, R3, R1 > 29 Pr6 < R5, R6, R7, R8, R9, R1, R2, R3, R4 > 29 Pr7 < R9, R8, R7, R6, R5, R4, R3, R2, R1 > 30 Ranked individuals w. r. t. disagreement Pr1 = < R1, R3, R2, R4, R5, R6, R7, R8, R9 > vs Pr2 = < R2, R3, R4, R1, R5, R8, R6, R7, R9 > TIE PAIRS Pr1, Pr2 (R1, R3), (R2, R3), (R6, R8), (R7, R8) Candidate pairs to be asked to the decision maker Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 23.
    Our Approach: UserFeedback Using Interactive Genetic Algorithm for Requirements Prioritization 23 Why ties appear in population? (i) Contradictory information w.r.t. initial constraints (ii) Nothing is expressed explicitly in initial constraints (iii) Common Domain Knowledge but different positions in the individuals Simple example: Why (R7, R8)? Case I: Contradictory w.r.t. Prio & Dep.. Why (R2, R3)? Case III: Common knowledge but different positions TIE PAIRS Pr1, Pr2 (R1, R3), (R2, R3), (R6, R8), (R7, R8) Prio Dep Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 24.
    Our Approach: UserFeedback Using Interactive Genetic Algorithm for Requirements Prioritization 24 TIE PAIRS Pr1, Pr2 (R1, R3), (R2, R3), (R6, R8), (R7, R8) Eli User Preference Graph eliOrd Experience & knowledge ‘<‘ or ‘>’ So, user knowledge is playing important role… Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 25.
    Our Approach: NewRound with New Constraints Using Interactive Genetic Algorithm for Requirements Prioritization 25  The new evolved population after using GA operators on population:  Crossover  Mutation  Selection  is compared against the new set of constraints graphs Eli DepPrio Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 26.
    Using Interactive Genetic Algorithm for Requirements Prioritization 26 Evaluation ofthe Approach Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 27.
    The Case Study Using Interactive Genetic Algorithmfor Requirements Prioritization 27  Prioritize requirements for a real software system, as part of the project ACube (Ambient Aware Assistance) designing a highly technological monitoring environment to be deployed in nursing homes to support medical and assistance staff  After user requirements analysis phase, 60 user requirements and 49 technical requirements Four macro-scenarios have been identified. ID Macro-Scenario # of requirements FALL Monitoring falls 26 ESC Monitoring escapes 23 MON Monitoring dangerous behavior 21 ALL The three scenarios 49 *ACube is a social welfare project coordinated by Fondazione Bruno Kessler (FBK) and funded by Autonomous Province of Trento under Bando Grandi Progetti, 2006. Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 28.
    Gold Standard (GS) Using Interactive Genetic Algorithmfor Requirements Prioritization 28  For each of the four macro-scenarios, we obtained the Gold Standard (GS) prioritization from the Software Architect of the ACube project  The GS prioritization is the ordering given by the software architect to the requirements when planning their implementation during the ACube project. Why Gold Standard?  To measure disagreements with respect to GS.  To evaluate our approach in terms of disagreement against other non-interactive approaches using the same GS. Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 29.
  • 30.
    R Q 1 Using Interactive Genetic Algorithmfor Requirements Prioritization 30 RQ1 (Convergence) Can we observe convergence with respect to the finally elicited fitness function? - Convergence is not obvious immediately, as Eli Graph is evolving at early stages. - Although the full fitness function is known only at end of elicitation process. The best individual in each population converges toward a low value of the final fitness function.Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions YES!
  • 31.
    R Q 2 Using Interactive Genetic Algorithmfor Requirements Prioritization 31 RQ2 (Role of interaction) Does IGA produce improved prioritizations compared to non-interactive requirement ordering? IGA outperforms substantially GA (and RAND), especially when a higher number of pairwise comparisons can be carried out YES! Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 32.
    R Q 3 Using Interactive Genetic Algorithmfor Requirements Prioritization 32 RQ3 (Role of initial precedence constraints) How does initial availability of precedence constraints affect the performance of IGA? - Different type of Domain Knowledge affects IGA significantly - The improvement of IGA over GA is even higher when limited ranking information is available Improves! Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 33.
    R Q 4 Using Interactive Genetic Algorithmfor Requirements Prioritization 33 RQ4 (Robustness) Is IGA robust with respect to errors committed by the user during the elicitation of pairwise comparisons? IGA can tolerate user errors up to 20%. YES!Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 34.
    General Discussion Using Interactive Genetic Algorithm for Requirements Prioritization 34 Cost/benefit trade off offered by IGA as compared to AHP is extremely interesting  With an elicitation effort reduced to 10% of the one required by AHP, IGA produces an apprx. ordering which has a quite low disagreement from the requirement positions in the GS.  User Errors tolerance offered by IGA as compared to AHP is highly reasonable  With an elicitation of very significant amount of less pairs than AHP (i.e. pairwise comparisons), it is reasonable even if IGA accepts up to 20% user error, while using AHP even 10% user error can result a total bad ordering. Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 35.
    Conclusions & FutureWorks Using Interactive Genetic Algorithm for Requirements Prioritization 35  We proposed an Interactive Genetic Algorithm to collect pairwise information useful to prioritize the requirements for a software system.  We also verified the robustness of the algorithm with respect to increasing user feedback errors.  We evaluated the approach in a real project (ACube).  In summary, we contributed A NOVEL APPROACH to prioritize requirements & tested its effectiveness empirically. What’s Next? Algorithm: - refining the algorithm - improving GA operators Experiment - off-line: comparisons with other approaches - on-line: controlled experiments with real object (i.e. human) Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 36.
    Paper Publication Using Interactive Genetic Algorithm for Requirements Prioritization 36 Apaper was published from this work and was presented at SSBSE 2010: Paolo Tonella1, Angelo Susi1, and Francis Palma2, "Using Interactive GA for Requirements Prioritization" in 2nd International Symposium on Search Based Software Engineering 2010. 1Fondazione Bruno Kessler, Software Engineering Research Unit; 2Department of Inf. Eng. and Computer Science, University of Trento. Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions
  • 37.
    Using Interactive Genetic Algorithm for Requirements Prioritization 37 Thank You! Outline Problem RelatedWorks Genetic Algo Our Approach Case Study Results Discussions Conclusions