Other Estimation
Techniques
Part III
Mel Rosso-Llopart
Carnegie Mellon University

Rosso-Llopart ©2013

1
Agenda


Other Estimation on estimation
−
−



Techniques
Exercises on estimation

Improving your Estimates
−



Case s...
3
©
D
2
G
I
L
M
2
E
T
O
,
L
L
C

Session Objectives


Define what an estimate is one more time






Definitions have ...
4
©
D
2
G
I
L
M
2
E
T
O
,

What is an estimate?


Estimate noun [C] “an approximate
calculation or judgement of the size,...
5
©
D
2
G
I
L
M
2
E
T
O
,
L
L
C

How You Can Tell A Good Estimation
Technique
You can teach it
 You can use it in all pha...
Estimate uncertainty
4x

2x

x

Feasibility Requirements

Design

Code

Delivery

0.5x

0.25x

Rosso-Llopart ©2013

6
Inputs and Outputs to the
Estimation Process
Classical view of software estimation process [Vigder/Kark]

Product Specs
Si...
Productivity
The output-input ratio within a time period with due
consideration for quality.
 Standard Productivity = out...
Why is this important?
1) Time is part of the productivity relationship.
2) Time is a very important part with the exponen...
Basic estimation process



First figure out what are the similarities
Then look for differences
−

Use these to determi...
First look for “similarities”




Have we done something similar
Do we have data on that project
How long ago was it
−
...
Next look for “differences”



Do we understand the differences
Do we have expertise in this new area
−
−



Training c...
Function points
Function

point count modified by
complexity of the project
FPs can be used to estimate LOC
depending on...
Function Point Estimation
Step One – System Size
System Elements and their Complexity
Description

Low

Medium

High

Tota...
Function Point Basics


Count the following:

inputs
 outputs
 inquiries – Get info only
 logical internal files – Sav...
Function Points
(14 factors)


Compute the technical complexity
factor (TCF)
 Assign a value from 0 (“not
present”) to 5...
Converting Function Points to Lines of Code
Language

LOC/Function Code Point

Access
C
C++
COBOL
Excel
HTML
JAVA
Javascri...
Function Points
FP

are dimensionless numbers
FP can have multiple uses as metrics:
pages/FP
 defects/FP
 Cost/FP
 LO...
The Downside of Function Points
 Difficult

to be consistent across the set of
counters
 Does not reflect internal compl...
Algorithmic code modelling
A

formulaic approach based on historical
cost information and which is generally
based on the...
Putnam’s Software LIfe-cycle Model (SLIM)
 Putnam

used Software and
manpower-build Equation
 This method makes use of
a...
2
©
D
2
1
G
I
L
M
3
E
T
O
,
L
L
C

Most Common Size Estimation
Techniques
The WAG
 Estimation by analogy




Small Grai...
2
©
D
3
1
G
I
L
M
3
E
T
O
,

WAG - Wild Altogether Guess



L
L
C






Totally new environment
No historical data
N...
2
©
D
4
1
G
I
L
M
3
E
T
O
,
L
L
C

Estimation by analogy
 The

method:



The project manager identifies the
values of c...
2
©
D
5
1
G
I
L
M
3
E
T
O
,
L
L
C

Estimation by analogy (2)


Advantages


Data is derived in an easily understandable ...
2
©
D
6
1
G
I
L
M
3
E
T
O
,
L
L
C

Estimation by analogy (3)


Disadvantages


You need a historical database that conta...
2
©
D
7
1
G
I
L
M
3
E
T
O
,
L
L
C

Experts judgment


Expert judgment involves consulting with human experts
to use their...
2
©
D
8
1
G
I
L
M
3
E
T
O
,
L
L
C

Parametric Or Algorithmic Methods


The algorithmic method uses equations to
create so...
2
©
D
9
1
G
I
L
M
3
E
T
O
,

Technique comparison
Method
Algorithmic Model

Strengths





L
L
C

Expert judgment



W...
Case Study Discussion

Estimation as a Tool

Rosso-Llopart ©2013

30
Discussion Questions








What are the major decisions that the manager
must make to help him with his estimation
p...
Questions?
Comments?

Rosso-Llopart ©2013

32
Improving your
Estimation Techniques
Mel Rosso-Llopart
Carnegie Mellon University

Rosso-Llopart ©2013

33
3
©
D
4
2
G
I
L
M
2
E
T
O
,

As a reminder, an estimate is:


Estimate noun [C] “an approximate
calculation or judgement ...
Agenda


Metrics and Estimation



Cost estimation revisited



Process maturity

Rosso-Llopart ©2013

35
3
©
D
6
2
G
I
L
M
2
E
T
O
,

Metrics and Estimation


As you decide on your estimation process
Remember you need data for...
3
©
D
7
2
G
I
L
M
2
E
T
O
,
L
L
C

Scope of Software Metrics


Software Metrics have two main fields of
application:


M...
3
©
D
8
2
G
I
L
M
2
E
T
O
,
L
L
C

Software Product/Process
Measurement



An integral part of the Project Plan
Aims to ...
3
©
D
9
2
G
I
L
M
2
E
T
O
,
L
L
C

The Need for Software Metrics


Software Production suffers from
Excessive Costs
 Low...
4
©
D
0
2
G
I
L
M
2
E
T
O
,
L
L
C

The Need for Software Metrics....


Software production suffers...because we
fail to:
...
Cost Estimation Process (Revisited)
Product Specs
Size Drivers

Size
Estimator

Size
Schedule

Complexity
Constraints
Pers...
4
©
D
2
G
I
L
M
2
E
T
O
,
L
L
C

Cost Estimation Process


To estimate the Cost of a system
We can better know the cost i...
4
©
D
3
2
G
I
L
M
2
E
T
O
,
L
L
C

Software Size and Complexity
 Estimating/Measuring


Software Size

Measures must be:...
4
©
D
4
2
G
I
L
M
2
E
T
O
,
L
L
C

Software Metrics & Process Maturity
 Critical

activities for a mature process

Set go...
4
©
D
5
2
G
I
L
M
2
E
T
O
,
L
L
C

Define



Appropriate metrics for your group
Keep in mind that data collection costs ...
4
©
D
6
2
G
I
L
M
2
E
T
O
,
L
L
C

Summary




You should be able to define what metrics are
appropriate for software pr...
Questions?
Comments?

Rosso-Llopart ©2013

47
Wrap Up


Software Estimation Concepts (Agile)



Other Estimation techniques



Improving your estimation

Rosso-Llopa...
Software Estimation Concepts
Summary


What an estimate is and what it is not.



The basic estimation techniques for Ag...
Other Estimation Techniques
Summary


Function points – Father of many techniques
−

Use case Points

−

Object points

...
Improving your estimation
Summary


Using metrics and historical data to improve
estimation



What are good metrics


...
Questions?
Comments?

Rosso-Llopart ©2013

52
References






Mike Cohn, “Agile Estimation and Planning”,
Prentice Hall, 2006.
Watts Humphrey, “Introduction to the ...
Upcoming SlideShare
Loading in …5
×

Agile estimation 3_Мел Росс

598 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
598
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
13
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Agile estimation 3_Мел Росс

  1. 1. Other Estimation Techniques Part III Mel Rosso-Llopart Carnegie Mellon University Rosso-Llopart ©2013 1
  2. 2. Agenda  Other Estimation on estimation − −  Techniques Exercises on estimation Improving your Estimates −  Case study – Estimation as a tool Wrapup Rosso-Llopart ©2013 2
  3. 3. 3 © D 2 G I L M 2 E T O , L L C Session Objectives  Define what an estimate is one more time    Definitions have been given to give perspective on how many ways an estimate can viewed Define what metrics are and what metrics are appropriate for software projects to improve future estimates Cover what we have discussed make good estimates for software projects Rosso-Llopart ©2013 3
  4. 4. 4 © D 2 G I L M 2 E T O , What is an estimate?  Estimate noun [C] “an approximate calculation or judgement of the size, value, amount, cost, etc. of something” Cambridge Dictionary  So how do we improve our approximations? L L C Historical Data of course Rosso-Llopart ©2013 4
  5. 5. 5 © D 2 G I L M 2 E T O , L L C How You Can Tell A Good Estimation Technique You can teach it  You can use it in all phases of your work  You can use it on all components  You can use statistical evaluation of its results  You can extend it into the future  You can evaluate its accuracy  Rosso-Llopart ©2013 5
  6. 6. Estimate uncertainty 4x 2x x Feasibility Requirements Design Code Delivery 0.5x 0.25x Rosso-Llopart ©2013 6
  7. 7. Inputs and Outputs to the Estimation Process Classical view of software estimation process [Vigder/Kark] Product Specs Size Drivers Size Estimator Complexity Constraints Personnel Skill Environment Size Schedule Cost Estimation Process Effort Cost Project Estimate COST DRIVERS Rosso-Llopart ©2013 7
  8. 8. Productivity The output-input ratio within a time period with due consideration for quality.  Standard Productivity = outputs/inputs Software Productivity  Process Productivity = (Size in SLOC) / (Effort)^(1/3) x (Time)^(4/3)*  * Larry Putnam, QVC Rosso-Llopart ©2013 8
  9. 9. Why is this important? 1) Time is part of the productivity relationship. 2) Time is a very important part with the exponents saying it has four times the weight of Effort.   So the longer it takes the more impact on productivity. Is this a good or bad impact? Rosso-Llopart ©2013 9
  10. 10. Basic estimation process   First figure out what are the similarities Then look for differences − Use these to determine the effort needed Does effort equal progress? Amount of time invested equal progress? Rosso-Llopart ©2013 10
  11. 11. First look for “similarities”    Have we done something similar Do we have data on that project How long ago was it −  Do we still have the expertise −  Geometric loss of understanding Expertise does not last Do we have the artifacts from that project − Can we read them Rosso-Llopart ©2013 11
  12. 12. Next look for “differences”   Do we understand the differences Do we have expertise in this new area − −  Training cost time and money Can we get the expertise quickly Do we have a proxy for this difference − Have we done something similar on other projects Rosso-Llopart ©2013 12
  13. 13. Function points Function point count modified by complexity of the project FPs can be used to estimate LOC depending on a language factor LOC = AVC * number of function points  AVC is a language-dependent factor  FPs can be very subjective, depend on estimator  Automatic function-point counting maybe impossible Rosso-Llopart ©2013 13
  14. 14. Function Point Estimation Step One – System Size System Elements and their Complexity Description Low Medium High Total Inputs __x 3 __x 4 __x 6 ____ Outputs __x 4 __x 5 __x 7 ____ Queries __x 3 __x 4 __x 6 ____ Files __x 7 __x 10 __x 15 ____ Program Interfaces __x 5 __x 7 __x 10 ____ TOTAL UNADJUSTED FUNCTION POINTS Rosso-Llopart ©2013 ____ 14
  15. 15. Function Point Basics  Count the following: inputs  outputs  inquiries – Get info only  logical internal files – Save state  external interfaces  Apply “simple, average, complex” multipliers (Low, Medium, High)  Apply the 14 adjustment factors (such as designed for reuse? in a heavy traffic environment? etc.)  Rosso-Llopart ©2013 15
  16. 16. Function Points (14 factors)  Compute the technical complexity factor (TCF)  Assign a value from 0 (“not present”) to 5 (“strong influence throughout”) to each of 14 factors such as transaction rates, portability  Add 14 numbers ⇒ total degree of influence (DI)  TCF = 0.65 + 0.01 × DI Technical complexity factor (TCF) lies between 0.65 and 1.35  The number of function points (FP) is given by   FP = UFP × TCF Rosso-Llopart ©2013 16
  17. 17. Converting Function Points to Lines of Code Language LOC/Function Code Point Access C C++ COBOL Excel HTML JAVA Javascript Oracle Perl Powerbuilder SQL VBScript Visual Basic Web Scripts 35 162 66 77 47 47 62 58 30 60 32 40 36 47 44 Rosso-Llopart ©2013 Source: Quality Software Management (http://www.qsm.com/FPGearing.html) 17
  18. 18. Function Points FP are dimensionless numbers FP can have multiple uses as metrics: pages/FP  defects/FP  Cost/FP  LOC/FP  Rosso-Llopart ©2013 18
  19. 19. The Downside of Function Points  Difficult to be consistent across the set of counters  Does not reflect internal complexity very well  Adjustments needed Symons on complexity1  Reifer for real-time systems2   Must be converted to LOC; database needed for each language (Organizational)  There is hope: IFPUG – Release of counting standard, but you must be member 1 - Reifer, Donald J., Asset-R IFPUG Spring Conference, Baltimore, Maryland, April, 1991. 2 - Symons, Charles R., Mark II Function Points IEEE Transactions on Software Engineering, Vol. SE-14, no. 1, January 1988 Rosso-Llopart ©2013 19
  20. 20. Algorithmic code modelling A formulaic approach based on historical cost information and which is generally based on the size of the software  Examples are:  Putnam’s Software Life-cycle Model (SLIM)  COCOMO  COCOMO II  Advantages:  Its results are repeatable  Objectiveness  Disadvantages:  Some subjective input is needed Rosso-Llopart ©2013 20
  21. 21. Putnam’s Software LIfe-cycle Model (SLIM)  Putnam used Software and manpower-build Equation  This method makes use of a so-called Rayleigh curve to estimate project effort, schedule and defect rate  Supported by Quantitative Software Management (QSM) www.qsm.com Rosso-Llopart ©2013 21
  22. 22. 2 © D 2 1 G I L M 3 E T O , L L C Most Common Size Estimation Techniques The WAG  Estimation by analogy   Small Grain and Large Grain “All analogy methods require a local, idiosyncratic, database”  Expert experience
  23. 23. 2 © D 3 1 G I L M 3 E T O , WAG - Wild Altogether Guess   L L C     Totally new environment No historical data No expertise in this area No experts to contact Research turns up no information …Guess, but make sure you record this
  24. 24. 2 © D 4 1 G I L M 3 E T O , L L C Estimation by analogy  The method:  The project manager identifies the values of certain “similarity dimensions”  dimensions are derived from the software specification include application type, size of application, language used, etc.   The project manager compares with other “similar projects” for a final estimate
  25. 25. 2 © D 5 1 G I L M 3 E T O , L L C Estimation by analogy (2)  Advantages  Data is derived in an easily understandable way  Complex domain-knowledge is not required, since information depends on historical data  Avoids problems commonly associated with knowledge elicitation (i.e. erroneous estimation)  Can be applied at a very early stage in production
  26. 26. 2 © D 6 1 G I L M 3 E T O , L L C Estimation by analogy (3)  Disadvantages  You need a historical database that contains a large variety of past projects  The accuracy of this method is directly related to the size and quality of the database available  There is generally no guarantee that the historical information about any specific project is completely accurate  Any inaccuracies contained in the historical data will have an effect on the estimation for the current project
  27. 27. 2 © D 7 1 G I L M 3 E T O , L L C Experts judgment  Expert judgment involves consulting with human experts to use their experience and understanding of a proposed project to provide an estimate for the cost of the project.  The expert can factor in differences between past project experiences and requirements of the proposed project  The expert can also factor in project impacts caused by new technologies, applications, and languages  Expert judgment always compliments other estimation methodologies  the estimates are no better than the expertise and judgment of the expert (hard to document)
  28. 28. 2 © D 8 1 G I L M 3 E T O , L L C Parametric Or Algorithmic Methods  The algorithmic method uses equations to create software estimates  Advantages:  being able to generate repeatable results  easily modifying input data  easily refining and customizing formulas  understanding of the estimating methods as formulas can be analyzed
  29. 29. 2 © D 9 1 G I L M 3 E T O , Technique comparison Method Algorithmic Model Strengths    L L C Expert judgment  Weakness Subjective inputs Assessment of exceptional circumstances Calibrated to past, not future Objective, repeatable, analyzable formula Efficient, good for sensitivity analysis Objectively calibrated to experience  Assessment of representative, interactions, exceptional circumstance   No better than participants Biases, incomplete recall   Analogy  Based on representative experience  Level of experience Parkinson’s Law  Correlates with some experience  Reinforces poor practice Price to win  Often gets the contract  Generally produces large overruns. Top-down  System level focus Efficient  Less detailed basis Less stable More detailed basis More stable Fosters individual commitments   Bottom-up      May overlook system level costs Requires more effort
  30. 30. Case Study Discussion Estimation as a Tool Rosso-Llopart ©2013 30
  31. 31. Discussion Questions     What are the major decisions that the manager must make to help him with his estimation problem? What would you have done differently if you were in the manager’s place? What estimation techniques from the case can you identify that are being used? What information about estimation did you gather that you did not already know? Rosso-Llopart ©2013 31
  32. 32. Questions? Comments? Rosso-Llopart ©2013 32
  33. 33. Improving your Estimation Techniques Mel Rosso-Llopart Carnegie Mellon University Rosso-Llopart ©2013 33
  34. 34. 3 © D 4 2 G I L M 2 E T O , As a reminder, an estimate is:  Estimate noun [C] “an approximate calculation or judgement of the size, value, amount, cost, etc. of something” Cambridge Dictionary  So how do we improve our approximations? L L C Historical Data of course…and therefore metrics. Rosso-Llopart ©2013 34
  35. 35. Agenda  Metrics and Estimation  Cost estimation revisited  Process maturity Rosso-Llopart ©2013 35
  36. 36. 3 © D 6 2 G I L M 2 E T O , Metrics and Estimation  As you decide on your estimation process Remember you need data for it  You need to be able to teach it  You need everyone to understand it  L L C  Or it will fail to be used and improve what you do Rosso-Llopart ©2013 36
  37. 37. 3 © D 7 2 G I L M 2 E T O , L L C Scope of Software Metrics  Software Metrics have two main fields of application:  Measure/assessment of a software product or process   e.g. a baseline – where are we now Estimation of critical features involved in a software project (e.g. a future estimate)  …And the reason why we are looking at this topic Rosso-Llopart ©2013 37
  38. 38. 3 © D 8 2 G I L M 2 E T O , L L C Software Product/Process Measurement   An integral part of the Project Plan Aims to provide data which can be used to better evaluate (control) the production of software “You cannot control what you cannot measure” Tom de Marco (1982) Rosso-Llopart ©2013 38
  39. 39. 3 © D 9 2 G I L M 2 E T O , L L C The Need for Software Metrics  Software Production suffers from Excessive Costs  Low Productivity  Poor Quality  Lack of Standards, Professionalism  Poor estimates of when, what and how much  Rosso-Llopart ©2013 39
  40. 40. 4 © D 0 2 G I L M 2 E T O , L L C The Need for Software Metrics....  Software production suffers...because we fail to: Set measurable targets  Measure the ‘real’ costs in projects  Quantify the quality   Understand when it is “good enough” Properly evaluate new tools and techniques  Use our own estimates  Rosso-Llopart ©2013 40
  41. 41. Cost Estimation Process (Revisited) Product Specs Size Drivers Size Estimator Size Schedule Complexity Constraints Personnel Skill Environment Cost Estimation Effort Process Cost Historical Data Rosso-Llopart ©2013 Project Actuals Project Estimate 41
  42. 42. 4 © D 2 G I L M 2 E T O , L L C Cost Estimation Process  To estimate the Cost of a system We can better know the cost if we know the size of the system  Typical size measures are:  LOC – Lines of Code  FP – Function Points  Rosso-Llopart ©2013 42
  43. 43. 4 © D 3 2 G I L M 2 E T O , L L C Software Size and Complexity  Estimating/Measuring  Software Size Measures must be: Useful for planning  Precise  Automatically countable   Estimate must be:  Available early in the life-cycle  Estimation  of Effort Requires system complexity understanding  For example: COCOMO II Rosso-Llopart ©2013 43
  44. 44. 4 © D 4 2 G I L M 2 E T O , L L C Software Metrics & Process Maturity  Critical activities for a mature process Set goals  Measure progress  Evaluate whether the goals have been met   Metrics provide input for planning  measure the developed product  measure the development process  Rosso-Llopart ©2013 44
  45. 45. 4 © D 5 2 G I L M 2 E T O , L L C Define   Appropriate metrics for your group Keep in mind that data collection costs money  Rule of thumb < 3% of project cost Create a place to save data properly  Make it a habit to use data early and often in projects “Take care of schedule and budget will take care of itself” Adtranz project binder cover  Rosso-Llopart ©2013 45
  46. 46. 4 © D 6 2 G I L M 2 E T O , L L C Summary   You should be able to define what metrics are appropriate for software projects to improve future estimates Understand the need for estimates at many times during the software development projects Rosso-Llopart ©2013 46
  47. 47. Questions? Comments? Rosso-Llopart ©2013 47
  48. 48. Wrap Up  Software Estimation Concepts (Agile)  Other Estimation techniques  Improving your estimation Rosso-Llopart ©2013 48
  49. 49. Software Estimation Concepts Summary  What an estimate is and what it is not.  The basic estimation techniques for Agile − Planning poker − Time boxing  How to use the estimation data  Concept of relative size estimation Rosso-Llopart ©2013 49
  50. 50. Other Estimation Techniques Summary  Function points – Father of many techniques − Use case Points − Object points  None unit based estimation  Analogy estimation  Expert Judgement  Algorithmic modeling − Cocomo, Slim, others Rosso-Llopart ©2013 50
  51. 51. Improving your estimation Summary  Using metrics and historical data to improve estimation  What are good metrics  What do you look for in estimation techniques  Who should the data be for Rosso-Llopart ©2013 51
  52. 52. Questions? Comments? Rosso-Llopart ©2013 52
  53. 53. References    Mike Cohn, “Agile Estimation and Planning”, Prentice Hall, 2006. Watts Humphrey, “Introduction to the Personal Software Process”, Addison-Wesley, 1997. Steve McConnell, “Software Estimation Demystifying the Black Art”, Microsoft press, 2006. Rosso-Llopart ©2013 53

×