Experimental Study Using Functional Size Measurement in Building Estimation Models for Software Project Size

779 views
656 views

Published on

This paper reports on an experiment that investigates the predictability of software project size from software product size. The predictability research problem is analyzed at the stage of early requirements by accounting the size of functional requirements as well as the size of non-functional requirements. The experiment was carried out with 55 graduate students in Computer Science from Concordia University in Canada. In the experiment, a functional size measure and a project size measure were used in building estimation models for sets of web application development projects. The results show that project size is predictable from product size. Further replications of the experiment are, however, planed to obtain more results to confirm or disconfirm our claim.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
779
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
31
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Experimental Study Using Functional Size Measurement in Building Estimation Models for Software Project Size

  1. 1. Nelly Condori-Fernandez, Luigi Buglione, Maya Daneva, Olga Ormandjieva SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 1
  2. 2. Goals of the presentation: presentation  G1. Discuss the estimation process in a software project, moving from initial requirements and their inner nature  G2. Propose a possible hybrid approach for improving such process, mixing two different viewpoints on software, looking both at the project as well as the product entities  G3. Measure in a controlled case study the effectiveness of such approach, noting possible issues for next improvements SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 2
  3. 3. Agenda x Software Requirements  FR vs NFR  NFR: Non-Functional Requirements x Sizing & Estimating  Why the need for sizing requirements?  The cone of uncertainty  Possible approaches  Project management approach  Project Size Unit (PSU)  Product functional approach  FSMM  COSMIC (ISO/IEC 19761) x A possible approach  Predicting CFP with PSU  Scale types  Refined relationships between CFP and PSU x Results  Context and sample projects  Student’s Project Historical Data x Conclusions & Prospects SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 3
  4. 4. Software Requirements FR vs NFR • Q: what is a requirement? A: “A software capability that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed documentation” [Leffingwell & Widrig, 2003] • General types of requirements: • Functional Requirements (FR) • Non- Functional Reqs (NFR)  ...For each shot the system  ..The response time shall be shall notify the players whether no more than 1 seconds for 95% the shot was a hit or miss... of responses and no more than 2 seconds for the remaining responses... SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 4
  5. 5. Software Requirements NFR: Non-Functional Requirement • In literature, NFRs are referred to as…  -ilities  Constraints  Quality attributes  Quality of service requirement  More? • IEEE-STD 830-1998 defines NFR as…  …“software requirement that describes not what the software will do, but how the software will do it…”  More definitions? • Nature of NFRs…  Subjective: viewed and interpreted differently by different people  Relative: interpretation and importance vary depending on the considered system  Interacting: attempts to achieve one NFR can hurt or help achievement of other  Global and scattered: one NFR affects multiple functionalities, or the whole system  More? SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 5
  6. 6. Agenda x Software Requirements  FR vs NFR  NFR: Non-Functional Requirements x Sizing & Estimating  Why the need for sizing requirements?  The cone of uncertainty  Possible approaches  Project management approach  Project Size Unit (PSU)  Product functional approach  FSMM  COSMIC (ISO/IEC 19761) x A possible approach  Predicting CFP with PSU  Scale types  Refined relationships between CFP and PSU x Results  Context and sample projects  Student’s Project Historical Data x Conclusions & Prospects SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 6
  7. 7. Sizing & Estimating Why the need for sizing FRs and NFRs? • Effort is a function of Size • Most effort estimation models use size as input for cost estimation  Most widely used metric of the size of a finished system is source lines of code (SLOC), delivered source instructions (DSI)  Two metrics of size applicable from the requirements specification phase are COSMIC Function Points (CFP) and Project Size Unit (PSU) • The cone of uncertainty Source: Boehm B., Software Engineering Economics, Prentice-Hall, 1981 SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 7
  8. 8. Sizing & Estimating Entities to be measured: STAR taxonomy Organization/ SBU Project Resources Process Product fsu (e.g. UFP, CFP, …) Measurement SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 8
  9. 9. Sizing & Estimating History of Functional Size Measurement Methods (FSMM) MkII FPA 1.3 COSMIC-FFP – MkII ISO/IEC 19761 FPA NESMA IFPUG IFPUG 4.1 Allan 4.0 Albrecht FPA 1980 1985 1990 1995 2000 Note: recently also FISMA FPA become an ISO International Standard (IS 29881:2008) SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 9
  10. 10. Sizing & Estimating COSMIC Measurement Method « Front « Back end » BOUNDARY end » USERS SOFTWARE ENTRIES STORE PERSISTENT DATA EXITS (‘WRITE’) Hardware DATA MANIPULATION Storage OR TRANSFORMATION or ENTRIES Engineered RETRIEVE PERSISTENT DATA Devices EXITS (‘READ’) SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 10
  11. 11. Sizing & Estimating Some thoughts on measurable entities… Container  …an 'application' (software) is not the project, (Project) therefore it cannot be represented in size terms only by a product metric such as FP (generically, as fsu).  …the 'container' (the project) is larger than its 'content' (the product). …therefore, how could it be possible to size the bigger entity by the littler one and therefore to make all the subsequent technical and economical Content assumptions on such size unit? (Product) …a possible answer could be to consider at the same time (or at least, in different moments during the SLC lifetime) sizing measures for different entities (project, product) SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 11
  12. 12. Sizing & Estimating Some thoughts on measurable entities…is something missing? • Rationale: is the productivity ratio (as now applied) meaningful or not?  Overall productivity is underestimated (no “Quality | Technical points”) on the upper part of the formula to counterbalance the overall project effort on the lower part  Project Size: the size of a software project, derived by quantifying the (implicit/explicit) user requirements refereable to the scope of the project itself SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 12
  13. 13. Sizing & Estimating Project Size Unit (PSU) • Project Size Unit (PSU)  Origin: created in 2003, it’s a PM-based technique for taking care of all possible user requirements – no matter the type – within the project framework  Goal: to create a virtual sizing unit for the whole project  Logical entity to count: WBS tasks, classified by several criteria  Weights: complexity weights by effort ranges (periodically revised)  Typical usage: internal (process improvement), but also external when the weighting system is stable among stakeholders (benchmarking)  Input: the IFPUG UFP formula  URL: www.semq.eu/leng/sizestpsu.htm PSU = ∑ ∑ taski * weight j i = M ,Q ,T j = H ,M , L • Application Scope  Projects, not only those for software ones (NewDev, Enh)  Project: “a temporary endeavor undertaken to create a unique product, service, or result” (PMBOK2008, Glossary) SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 13
  14. 14. Sizing & Estimating Project Size Unit (PSU): an example 1. Define HLR and refine them into RHLR UR 2. Translate RHLRs into WBS’s tasks, assigning an effort 3. Classifying tasks per type (M/Q/T), SLC phase and complexity SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 14
  15. 15. Sizing & Estimating Project Size Unit (PSU): an example 4. Count tasks frequencies by type, SLC phase and complexity 5. Count PSU 6. Compute effort distribution by task type (M/Q/T) and SLC phase SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 15
  16. 16. Sizing & Estimating Sizing measures and possible gathering moments in the SLC SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 16
  17. 17. Agenda x Software Requirements  FR vs NFR  NFR: Non-Functional Requirements x Sizing & Estimating  Why the need for sizing requirements?  The cone of uncertainty  Possible approaches  Project management approach  Project Size Unit (PSU)  Product functional approach  FSMM  COSMIC (ISO/IEC 19761) x A possible approach  Predicting CFP with PSU  Scale types  Refined relationships between CFP and PSU x Results  Context and sample projects  Student’s Project Historical Data x Conclusions & Prospects SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 17
  18. 18. A possible approach Predicting CFP with PSU • Early Size (in CFP) prediction of FR and NFR determines the likely future values of product size based on existing PSU measure of the same product • Advantages 1. allow for accurate size prediction of all FR and NFR, including those which are not (yet) stated in measurable terms 2. reduces the size measurement effort at this early stage • Theoretical aspect 1. Analyze the scale types of PSU and CFP 2. Given the lowest scale type, identify the corresponding to it type of admissible transformation (relation) between both units of measurement • Empirical aspect 1. Use available project data on PSU, CFP and Effort 2. Derive the prediction formula SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 18
  19. 19. A possible approach Scale type and admissible transformations Scale type Admissible transformations Examples Nominal 1:1 mapping from M to M’ Labeling, classifying entities Ordinal Monotonic increasing function Preference, hardness, air quality, intelligence test (raw scores) from M to M’, that is, M(x) ≥ M(y) implies M’(x) ≥ M’(y) Interval M’=aM + b(a>0) Relative time, temperature (Fahreneit, Celsius), intelligence tests (standardized scores) Ratio M’=aM (a>0) Time interval, lenght, temperature (Kelvin) Absolute M’=M Counting entities Source: Fenton N. & Pfleeger S.L., Software Metrics: A Rigorous and Practical Approach, 2° ed., Course Tech., 1998, ISBN 978-0534954253 SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 19
  20. 20. A possible approach Analysis of scale types of PSU and CFP • PSU: at least interval scale type  respects the additive property • CFP: at least ratio scale  Ratio of two values is meaningful • Both are valid size measurement units   theoretically there should exist an admissible transformation of type PSU = k * CFP + b(k > 0) SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 20
  21. 21. A possible approach Refined relationships between CFP and PSU • Relationships between PSU and CFP PSU f = k1* CFPf + b1 PSU n f = k 2 * CFPnf + b 2 • PSU and CFP respect the additive property PSU = PSUf + PSU nf CFP = CFPf + CFPnf SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 21
  22. 22. Agenda x Software Requirements  FR vs NFR  NFR: Non-Functional Requirements x Sizing & Estimating  Why the need for sizing requirements?  The cone of uncertainty  Possible approaches  Project management approach  Project Size Unit (PSU)  Product functional approach  FSMM  COSMIC (ISO/IEC 19761) x A possible approach  Predicting CFP with PSU  Scale types  Refined relationships between CFP and PSU x Results  Context and sample projects  Student’s Project Historical Data x Conclusions & Prospects SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 22
  23. 23. First Results Experimental design: Research Questions and Variables 1. Null hypothesis, H10. Project Functional Size determine estimated from RQ1: In what extent does the product non-functional sizecannot be the project size? Product Functional Size. 2. Null hypothesis, H20. product functional size Size cannot be estimated RQ2: In what extent does the Project Non-Functional determine the project size? from Product Non-Functional Size. Type of Variables Response variable Product size (calculated by the students) (Dependent) Project size (calculated by an expert) Factor (Independent) Project size measurement method: PSU Product size measurement method: COSMIC and COSMIC-NFSM Parameters - Application domain (web-application domain), - Experience using size measurement methods, - Quality of requirements specification. SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 23
  24. 24. First Results Experimental Procedure 55 third-year students enrolled at Concordia University (Montreal, Group2 Group 3 Group 11 Group 1 Canada). Documentation Measurement The experiment was Applying CFP and NFSM organised as mandatory part of the “Software Requirement, Measurement” course Design and Analysis Applying PSU Work-Breakdown Structure Expert SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 24
  25. 25. First Results Context and sample projects • Measurement Process:  Moving by the same problem statement, each team had to: 1. estimate the effort in man/hours on the info available at the feasibility stage 2. calculate the two sizes (CFP, PSU) 3. realizing the web project 4. determine the actual effort at the project closure stage 5. calculate the two final sizes (CFP, PSU) • Size Units adopted:  COSMIC Measurement Manual, v3.0  Layers: Application layer  Perspective used: end user  PSU Measurement Manual, v1.21  4 complexity effort ranges based on projects’ data  H/MH/ML/L SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 25
  26. 26. First results Students’ Project Historical Data SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 26
  27. 27. First results Students’ Project Historical Data & Regression analysis Hypothesis H10 is not rejected: Null significance, p= 0.24 However, excluding two data points from the analysis: PSU FUR = 1.003 * CFP − 29.897 FUR SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 27
  28. 28. First results Students’ Project Historical Data & Regression analysis Hypothesis H20 is rejected: Medium significance, p= 0.033 PSU NFR = 4.61* CFPNFR + 29.04 SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 28
  29. 29. Agenda x Software Requirements  FR vs NFR  NFR: Non-Functional Requirements x Sizing & Estimating  Why the need for sizing requirements?  The cone of uncertainty  Possible approaches  Project management approach  Project Size Unit (PSU)  Product functional approach  FSMM  COSMIC (ISO/IEC 19761) x A possible approach  Predicting CFP with PSU  Scale types  Refined relationships between CFP and PSU x Results  Context and sample projects  Student’s Project Historical Data x Conclusions & Prospects SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 29
  30. 30. Conclusions & Prospects • Main Issue/Goal observed  Improve predicatibility of project effort estimation as earlier as possible by obtaining the lowest ARE/MREas possible • State-of-the-art  Typical usage of single sizing & estimation methods (e.g. expert-based, analogy, parametric-based), covering a single perspective per time • Possible solution  Use at least two sizing methods according to their pros&cons during the whole SLC phases • Challenge…  Predicting CFP of FUR and (more importantly) NFR moving from project scope knowledge captured in PSU estimates • Possible advantages  Early productivity analysis from the predicted CFP size  Such solution can be automated within a PM tool (e.g. MS-Project) • Next actions/Prospects  A wider application of such approach on a larger number of projects, in order to validate it and stress eventual weakenesses SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 30
  31. 31. Thanks for your attention ! SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 31
  32. 32. Nelly Condori-Fernandez <nelly@pros.upv.es> Luigi Buglione: <luigi.buglione@eng.it> Maya Daneva: <m.daneva@utwente.nl> Olga Ormandjieva: <ormanj@cse.concordia.ca> SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 32

×