Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Improving the cosmic approximate sizing using the fuzzy logic epcu model alain abran

246 views

Published on

Presentation from the IWSM Mensura 2015 conference held October 5-7 in Cracow, Poland

Published in: Software
  • Login to see the comments

  • Be the first to like this

Improving the cosmic approximate sizing using the fuzzy logic epcu model alain abran

  1. 1. Improving the COSMIC Approximate Sizing using the Fuzzy Logic EPCU Model Francisco Valdés Souto & Alain Abran École de Technologie Supérieure fvaldes@ciencias.unam.mx alain.abran@etsmtl.ca 1© 2015 Valdés-Souto & Abran
  2. 2. The Sizing Problem FSM methods work best when the information to be measured– is fully known. Early phases: only non detailed information is available. 2 UC Identification © 2015 Valdés-Souto & Abran FP Identification
  3. 3. Approximate size measurement • Not all documentation is available • Quality of documentation is poor • Portfolio decisions Frank Vogelezang, 1er Congreso Nacional de Medición y Estimación de Software, México 2015
  4. 4. Estimation (Approximation) New approaches Judge & Compute Current approximation approaches COMPUTE COSMIC standard Count Steve McConnell - Software Estimation, Demystifying the Black Art
  5. 5. Henderson et al.: investigated the relationship between FP & KLOC Meli: A. Early Function Points (EFP), based on IFPUG 4.0, B. Extended FP (XFP): EFP & 3 correction factors. Desharnais et al.: analysed 2 techniques: Function Points Simplified (FPS) a& Backfiring Conte et al.: Early & Quick (E&Q) COSMIC - more tests needed to adjust or to confirm it Vogelezang et al. : study of 50 projects to define size bands using the quartile approach. Santillo: Analytic Hierarchy Process, for making choices among alternatives Related Works on Approximation of Functional Size 1992 1997 2003 2004 2007 2011 2012 2013 5 Valdés et al. proposed a solution using the fuzzy logic model from [3-5], referred to as the EPCU model Almakadmeh: A framework to assign scaling factors for identifying the level of granularity of functional requirements specifications. The state of the art on approximate COSMIC FSM was discussed at IWSM/MENSURA 2013 © 2015 Valdés-Souto & Abran
  6. 6. Needs local calibrati on Requireme nt granularity level Consideration Early & Quick COSMIC approximation X Multilevel Approach (*) The precision of the method is strongly dependent on the training and capability of the practitioners who use it to understand the categories at higher levels of granularity. [38], this approximation approach combines scaling and classification approaches. Quick/Early X Use Cases The precision is directly proportional to the level of granularity of the analyzed use cases model. EPCU approximation approach Functional Process & Use Cases Does not require local calibration (less expensive ) and is useful when there are no historical data available. Needs local calibrati on Requireme nt granularity level Consideration Average Functional Process X Functional Process This approximation is valid as long as there is sufficient reason to assume that the sample on which the size of the average functional process is calculated is representative for the software of which the functional size of which size is approximated. [38] Fixed Size Classification X Functional Process This approximation is valid as long as there is sufficient reason to assume that the assigned size classification is representative for the software of which the functional size of which size is approximated. [38] Equal Size Bands approximation X Functional Process This method is recommended for the approximate sizing of software where the distribution of the functional process sizes is skewed. For the business application this method has little added value over the average functional process method (1) or the fixed size classification method (2). [38] Average Use Case approximation X Use Case This approximation is valid as long as there is sufficient reason to assume that the assigned size classification of an average use case is representative for the software of which the functional size of which size is approximated. [38] Approximate Sizing Approaches are based on 2 main assumptions: 1. Historical data exist for calculating the scaling factor (average, or size bands). 2. The whole set of requirements is described, or at least there is a commitment, defined by the requirements, about the scope of the software to be developed . Approximation Techniques Analysis Highlights
  7. 7. Original Equal Size Bands approach in Related Topics • Historical data set: – 37 business application development projects, each having a total size greater than 100 CFP.(Vogelezang , 2007): • Quartile values of the Functional Process from this dataset: Small = 4.8 CFP, Medium =7.7 CFP, Large = 10.7, and Very Large = 16.4 CFP 7 © 2015 Valdés-Souto & Abran
  8. 8. Analyzing the Dataset #Projects Sector Project Range Project Average Functional Size #FP Size 26 Banking 11-2743 476 1345 12375 8 Government 64-2364 481 838 3845 6 Insurance 84-1311 551 342 3305 7 Logistics 193-1164 538 321 3766 % FP included Description Average Value Q1 Small FP's 55% contains FP's in the range up to 6 CFP 3.7 Q2 Medium FP's 26% contains FP's in the range 6-10 CFP 7.7 Q3 Large FP's 14% contains FP's in the range 10-25 CFP 14.6 Q4 Very Large FP's 5% contains FP's of 25 CFP and larger 44.1 Quartile Table 3. Q-Size considering four sectors Table 2. Dataset Characterization
  9. 9. Defining new Output Variable domain considering this Dataset • Quartile values of the Functional Process considering Q-Size (Table 3) from this dataset: Small = 3.7 CFP, Medium =7.7 CFP, Large = 14.6, and Very Large = 44.1 CFP 9 © 2015 Valdés-Souto & Abran Figure 3. Output Variable Schema
  10. 10. COSMIC Approximate Sizing Using the EPCU Model • Variable 1: Perception of the size of the Use Case (subjective, experience-based). • Variable 2: The number of Objects of Interest related to the Use Case (subjective, experience-based). Functional Size Estimated for each Use Case (CFP) What variables influence the size of a Use Case? What is the possible range for the output variable? Dataset analysis 10 © 2015 Valdés-Souto & Abran
  11. 11. Experiment Design ALFA software system/ 14 Use Case descriptions EPCU Model 1- Define Input variables & membership functions, 2- Define the Inference rules between the input variables & output variable (Functional Size in CFP) 3-Define Output variable & membership functions 2. Selecting a Measurement Reference 3. Knowing the ALFA Software System 4. Data Collection 5. Data Analysis 1. Define de EPCU Context for Approximate Sizing Participants provided only with the ALFA list of Use Cases: assign values to the 2 input variables, based on their experience. 11 © 2015 Valdés-Souto & Abran
  12. 12. The same Case Study in 2012 • Case Study with 8 practitioners: – not familiar with the COSMIC method, – with no historical data for approximating the FSM using COSMIC, – did not know the EPCU model, – did not participate in the definition of the EPCU context. The only information available had = – A form with the list of Use Cases – Their own experience with the business process related to the project • The Case Study= a simulation of the early size estimation step with both approaches (Equal size band & Fuzzy Logic). 12 © 2015 Valdés-Souto & Abran
  13. 13. Case Study Data & Analysis 2014-2015 © 2012 Valdés-Souto & Abran 13 Practitioner Reference Functional Size in CFP Estimated Functional Size using the ‘Equal Size Bands’ MRE Estimated Functional Size using EPCU (range from 2 to 16.4) MRE Estimated Functional Size using EPCU Improved (range from 2 to 44) MRE Practitioner 1 250 81.7 67% 186.32 25% 430.76 72% Practitioner 2 250 93.3 63% 132.76 47% 240.74 4% Practitioner 3 250 84.6 66% 62.19 75% 81.65 67% Practitioner 4 250 93.3 63% 114.34 54% 190.33 24% Practitioner 5 250 105.2 58% 168.13 33% 379.86 52% Practitioner 6 250 81.7 67% 111.26 55% 183.14 27% Practitioner 7 250 93.5 63% 130.43 48% 240.91 4% Practitioner 8 250 114 54% 199.82 20% 493.47 97% MMRE 63% 45% 43% SDMRE 5% 18% 34% 2014 2015
  14. 14. Case Study Data & Analysis 2014-2015 © 2012 Valdés-Souto & Abran 14
  15. 15. Case Study Data Analysis • MMRE = 45% & SDMRE= 18%. • Maximum MMRE = 75% & Minimum MMRE = 20%. • Always underestimate Fuzzy Logic EPCU model & the real value (2014) Fuzzy Logic EPCU model & the real value (2015) • MMRE = 43% & SDMRE= 34%. • Maximum MMRE = 97% & Minimum MMRE = 4%. • Equally likely to be above or below the real value 15 © 2015 Valdés-Souto & Abran Using a cutoff about 16.4 CFP (2012, 2014) the approximation of functional size is underestimating; using the cutoff about 44 CFP (2015) , the results are above and below from the real value, as discussed by De Marco “An estimation is a prediction that is equally likely to be above or below the actual result.”
  16. 16. Exploratory Research Observations • Fuzzy Logic approach: – it does not use bands, but rather a continuous range in ε R, which is represented by a membership function. – But it is sensitive to min-max values Large scale experiments needed with: • More case studies • More participants for each case study • Increase the collection of a set of projects with their use cases or their functional process identified, in order to conduct a more in depth analysis of the EPCU Improved Size Approximation approach 16 © 2015 Valdés-Souto & Abran
  17. 17. Questions? 17 © 2015 Valdés-Souto & Abran
  18. 18. © 2012 Valdés-Souto & Abran 18 Use case Use case classificatio n(linguistic values) Use case size (value assignment ) Presence (level, not the number of) of object of interest related to the Use case classificatio n(linguistic values) Presence (level, not the number of) of object of interest related to the Use case (value assignment ) Estimated Functional Size using ‘EqualSize Bands’ approach Estimated Functional Size using EPCU(TO 16.4) Estimated Functional Size using EPCU(TO 44) Use case classification (linguistic values) Use case size (value assignme nt) Presence (level, not the number of) of object of interest related to the Use case classificati on (linguistic values) Presence (level, not the number of) of object of interest related to the Use case (value assignme nt) Estimated Functional Size using ‘Equal Size Bands’ approach Estimated Functional Size using EPCU (TO 16.4) Estimated Functional Size using EPCU (TO 44) Use case classificati on (linguistic values) Use case size (value assignme nt) Presence (level, not the number of) of object of interest related to the Use case classificati on (linguistic values) Presence (level, not the number of) of object of interest related to the Use case (value assignme nt) Estimated Functional Size using ‘Equal Size Bands’ approach Estimated Functional Size using EPCU (TO 16.4) Estimated Functional Size using EPCU (TO 44) Use case classificati on (linguistic values) Use case size (value assignme nt) Presence (level, not the number of) of object of interest related to the Use case classificati on (linguistic values) Presence (level, not the number of) of object of interest related to the Use case (value assignme nt) Estimated Functional Size using ‘Equal Size Bands’ approach Estimated Functional Size using EPCU (TO 16.4) Estimated Functional Size using EPCU (TO 44) Use case 1 MEDIUM 3 AVERAGE 4 7.7 16.4 44.0 MEDIUM 3 AVERAGE 3 7.7 12.7 27.5 MEDIUM 2 AVERAGE 2 7.7 6.6 8.8 SMALL 2 AVERAGE 2 4.8 6.6 8.8 Use case 2 MEDIUM 3 FEW 4 7.7 16.4 44.0 MEDIUM 3 AVERAGE 3 7.7 12.7 12.7 SMALL 1 FEW 1 4.8 1.6 2.0 MEDIUM 3 AVERAGE 2 7.7 8.4 11.6 Use case 3 SMALL 2 FEW 3 4.8 10.7 20.5 MEDIUM 3 AVERAGE 3 7.7 12.7 12.7 MEDIUM 2 FEW 1 7.7 4.6 5.3 MEDIUM 2 AVERAGE 2 7.7 6.6 8.8 Use case 4 SMALL 2 FEW 4 4.8 13.8 32.2 SMALL 1.5 FEW 1 4.8 2.6 2.9 SMALL 1 FEW 1 4.8 1.6 2.0 MEDIUM 3 MANY 3 7.7 12.7 27.5 Use case 5 SMALL 2 FEW 4 4.8 13.8 32.2 MEDIUM 3.2 AVERAGE 3.2 7.7 13.3 30.3 MEDIUM 2 AVERAGE 2 7.7 6.6 8.8 SMALL 2 AVERAGE 2 4.8 6.6 8.8 Use case 6 SMALL 2 AVERAGE 4 4.8 13.8 32.2 MEDIUM 3 AVERAGE 3 7.7 12.7 12.7 SMALL 1 FEW 1 4.8 1.6 2.0 MEDIUM 3 AVERAGE 3 7.7 12.7 27.5 Use case 7 MEDIUM 3 AVERAGE 3 7.7 12.7 27.5 SMALL 1.5 FEW 1.3 4.8 2.9 3.3 MEDIUM 3 AVERAGE 2 7.7 8.4 11.6 SMALL 2 FEW 2 4.8 6.6 8.8 Use case 8 SMALL 2 FEW 4 4.8 13.8 32.2 MEDIUM 3.3 AVERAGE 3 7.7 12.7 27.4 SMALL 2 FEW 2 4.8 6.6 8.8 MEDIUM 3 AVERAGE 3 7.7 12.7 27.5 Use case 9 SMALL 1 AVERAGE 3 4.8 7.9 10.5 SMALL 1 FEW 1.5 4.8 2.6 2.9 SMALL 1 FEW 1 4.8 1.6 2.0 SMALL 1 FEW 2 4.8 4.6 5.3 Use case 10 SMALL 2 FEW 3 4.8 10.7 20.5 SMALL 1.3 FEW 1 4.8 1.8 2.0 SMALL 1 FEW 1 4.8 1.6 2.0 MEDIUM 2 AVERAGE 3 7.7 10.7 20.5 Use case 11 MEDIUM 4 AVERAGE 3 7.7 12.5 26.4 MEDIUM 3 AVERAGE 3.5 7.7 14.6 35.7 SMALL 2 AVERAGE 2 4.8 6.6 8.8 MEDIUM 2 AVERAGE 2 7.7 6.6 8.8 Use case 12 SMALL 2 FEW 4 4.8 13.8 32.2 SMALL 1.4 FEW 1.4 4.8 3.2 3.9 SMALL 1 FEW 1 4.8 1.6 2.0 SMALL 2 FEW 2 4.8 6.6 8.8 Use case 13 MEDIUM 3 AVERAGE 4 7.7 16.4 44.0 MEDIUM 3 AVERAGE 3.2 7.7 13.4 30.4 MEDIUM 2 AVERAGE 2 7.7 6.6 8.8 MEDIUM 2 AVERAGE 2 7.7 6.6 8.8 Use case 14 SMALL 2 AVERAGE 4 4.8 13.8 32.2 MEDIUM 3.5 AVERAGE 3.5 7.7 14.7 36.5 MEDIUM 2 AVERAGE 2 7.7 6.6 8.8 MEDIUM 2 AVERAGE 2 7.7 6.6 8.8 81.7 186.3 430.8 93.3 132.8 240.7 84.6 62.2 81.7 93.3 114.3 190.3 Practitioner1 Practitioner2 Practitioner3 Practitioner4

×