IWSM Presentation

1. COSMIC Approximate Sizing
using a
Fuzzy Logic Approach -
A Case Study with
Industry Participants
Francisco Valdés Souto & Alain Abran
École de Technologie Supérieure
francisco.valdes.1@ens.etsmtl.ca
alain.abran@etsmtl.ca
1© 2014 Valdés-Souto & Abran

2. The Sizing Problem
FSM methods
work best when
the information
to be measured–
is fully known.
Early phases:
only non
detailed
information is
available.
2
UC
Identification
© 2014 Valdés-Souto & Abran
FP
Identification

3. Henderson et al.:
investigated the
relationship
between FP &
KLOC
Meli:
A. Early Function
Points (EFP), based
on IFPUG 4.0,
B. Extended FP (XFP):
EFP & 3 correction
factors.
Desharnais et al.:
analysed 2
techniques: Function
Points Simplified (FPS)
a& Backfiring
Conte et al.:
Early & Quick
(E&Q) COSMIC -
more tests
needed to
adjust or to
confirm it
Vogelezang et al. : study
of 50 projects to define
size bands using the
quartile approach.
Santillo:
Analytic Hierarchy
Process, for
making choices
among
alternatives
Related Works on
Approximation of Functional Size
1992 1997 2003 2004 2007 2011 2012 2013
3
Valdés et al. proposed
a solution using the
fuzzy logic model from
[3-5], referred to as the
EPCU model
Almakadmeh:
A framework to assign scaling factors
for identifying the level of granularity
of functional requirements
specifications.
The state of the art on approximate
COSMIC FSM was discussed at
IWSM/MENSURA 2013
© 2014 Valdés-Souto & Abran

4. 2nd Generation FSM method: COSMIC &
General Approach to Approximate Sizing
4© 2014 Valdés-Souto & Abran

5. Approximate Sizing Approaches
in the COSMIC Measurement Manual
Early sizing:
• for use early in the
life cycle of a project:
– before the
Functional User
Requirements (FUR)
are detailed and
specified.
Rapid sizing:
• for use when there is
not enough time to
measure the
required software
using the standard
method
5
© 2014 Valdés-Souto & Abran

6. Example 1: Average Functional Process approach.
Example 2: Fixed Size Classification approach.
Example 3: Equal Size Bands approach. (Vogelezang: Refined
Approximate or Quartile approach)
Example 4: Average Use Case approach.
Approximate Sizing Examples in the COSMIC
Advanced & Related Topics Manual
Each example based on 2 main assumptions:
1. Historical data exist for calculating the scaling factor (average, or
size bands).
2. The whole set of requirements is described, or at least there is a
commitment, defined by the requirements, about the scope of
the software to be developed.
6© 2014 Valdés-Souto & Abran

7. Example 3: Equal Size Bands approach
• Historical data set:
– 37 business application development projects, each having a total size
greater than 100 CFP.(Vogelezang , 2007):
• Quartile values of the Functional Process from this dataset:
Small = 4.8 CFP, Medium =7.7 CFP, Large = 10.7, and Very
Large = 16.4 CFP
7
© 2014 Valdés-Souto & Abran

8. 8
Fuzzy Logic EstimationModel
© 2014 Valdés-Souto & Abran

9. Roles in the Fuzzy Logic Model
The domain expert:
• Selects the types of input & output variables
• Selects the ranges of values for each variable
• Assigns the inference rules between the
inputs & output variables
The estimator (junior or expert):
• Select among the range of values of the input
variables
9
© 2014 Valdés-Souto & Abran

10. Roles in the Fuzzy Logic Model
The tool builder :
• Selects:
– the fuzzy logic maths options
• Ex. Trapezoidal shape, triangle shape, etc
– Fuzzification options
– De-fuzzication options
• Builds the estimation software tool as a shell
for the variables selected by the domain
experts
10
© 2014 Valdés-Souto & Abran

11. 11
Fuzzy Logic Estimation Model
F. Valdés & A. Abran - IWSM-Mensura 2007 (Palma de Mallorca, Spain)
Tool used for this exploratory research:
The fuzzy logic estimation model developed in the PhD
thesis of Francisco Valdes in 2011:
EPCU: Estimation of Projects in a Context of Uncertainty
Model initially tested with the estimation of projects duration
© 2014 Valdés-Souto & Abran

12. EPCU Model with Fuzzy Logic
Some characteristics of fuzzy logic estimation models
(Valdés et al., 2007, 2010, 2011):
1. Designed to deal with vague information (i.e. usually described by
linguistic variables).
2. Generates estimates with less dispersion than the experience-
based approach.
3. Enables a systematic replication: whatever the level of skills of the
people who assign the values for the input variables.
4. In the early phases (imperfect information environments) may be
preferable to the experience-based estimation approach, under
similar experimental conditions.
5. The performance of the EPCU estimation process for most of the
projects is significantly better than that of the experience-based
estimation approach, based on the quality criteria used.
12
© 2014 Valdés-Souto & Abran

13. COSMIC Approximate Sizing Using the
EPCU Model
• Variable 1:
Perception of the size of
the Use Case (subjective,
experience-based).
• Variable 2:
The number of Objects of
Interest related to the Use
Case (subjective,
experience-based).
Functional Size
Estimated for each
Use Case (CFP)
What variables influence the size
of a Use Case? What is the possible range for
the output variable?
(Vogelezang , 2007) 13
© 2014 Valdés-Souto & Abran

14. Case Study– Model Designer
A) Defines the Input variables with linguistic values
(fuzzy sets):
– Input variable 1: Size of Use Case= relative size from
small to very large
– Input variable 2: No. of objects of interest = Low, Average,
and High
• Domain of membership function: from 0 to 5 ε R.
B) Defines the Output variable with a:
– Min of a Use Case = 2 FP
– Max of a Use Case = 16.4 CFP
14
© 2014 Valdés-Souto & Abran

15. Case Study Participants
• Case Study with 8 practitioners:
– not familiar with the COSMIC method,
– with no historical data for approximating the FSM using COSMIC,
– did not know the EPCU model,
– did not participate in the definition of the EPCU context.
The only information available had =
– A form with the list of Use Cases
– Their own experience with the business process related to the project
• The Case Study= a simulation of the early size estimation step
with both approaches (Equal size band & Fuzzy Logic).
15
© 2014 Valdés-Souto & Abran

16. Experiment Design
ALFA software
system/ 14 Use
Case descriptions
EPCU Model
1- Define Input variables & membership
functions,
2- Define the Inference rules between
the input variables & output variable
(Functional Size in CFP)
3-Define Output variable & membership
functions
2. Selecting a Measurement Reference
3. Knowing the ALFA Software System
4. Data Collection
5. Data Analysis
1. Define de EPCU Context for
Approximate Sizing
Participants provided only with the ALFA list
of Use Cases: assign values to the 2 input
variables, based on their experience.
16
© 2014 Valdés-Souto & Abran

17. Participants tasks
A) Equal Size Band Model:
- Consider the UC as FP and Classify each of the 14
Use Cases from Small to Very Large (4 linguistic
categories)
B) Fuzzy logic model:
B1) Assign for each Use Case a relative size between
0 & 5
B2) Assign for each Use Case a relative number of
Objects of Interest between 0 & 5.
17
© 2014 Valdés-Souto & Abran

18. Case Study Data & Analysis
18
© 2014 Valdés-Souto & Abran

19. Case Study Data Analysis
• MMRE= 63% and SDMRE = 5%
• Maximum MMRE = 67% & Min MMRE =
54%.
Equal Size Bands approach & real value Fuzzy Logic EPCU model & the real value
• MMRE = 45% & SDMRE= 18%.
• Maximum MMRE = 75% & Minimum MMRE =
20%.
• Practitioners are not familiar with the COSMIC
method.
19
© 2014 Valdés-Souto & Abran

20. Exploratory Research Observations
• Participants did not know the COSMIC sizing method
• The Equal Size Band led to less accurate COSMIC sizes:
– With 9 participants for the same set of UC.
• The fuzzy logic model led to more accurate of size:
– The better results obtained could be associated to the use of use cases
instead of functional process, even though the use cases is at a higher
level of granularity than the functional processes.
20
© 2014 Valdés-Souto & Abran

21. Exploratory Research Observations
• Fuzzy Logic approach:
– it does not use bands, but rather a continuous range in ε R, which is
represented by a membership function.
– But it is sensitive to min-max values
Large scale experiments needed with:
• More case studies
• More participants for each case study
• Analyze the original data set of Equal Size Bands Approach in
order to define a “cut-off” instead to use the average for the
last band.
21
© 2014 Valdés-Souto & Abran

22. Questions?
22
© 2014 Valdés-Souto & Abran