Describe what the estimation is, sources of error and how to make estimation

1. 1
ESTIMATION
Author: Nguyen Phuc Hai
Created date: 1/9/2008

2. Agenda
2
What is estimation?
Reasons to make wrong estimations
Fundamental estimation Techniques
Estimation process of Agile Mobile
Politics, Negotiation, and Problem Solving

3. What is estimation
3
Estimates, Target and Commitments
Good estimation

4. Estimation
4
Estimation is a prediction of how long a project will
take or how much it will cost

5. Target
5
A Target is a statement of a desirable business
objective.
Examples:
quot;We need to have Version 2.1 ready to demonstrate at a
trade show in May.quot;
quot;We need to have this release stabilized in time for the
holiday sales cycle.“
Businesses have important reasons to establish targets
independent of software estimates. But the fact that a
target is desirable or even mandatory does not
necessarily mean that it is achievable.

6. Commitment
6
A commitment is a promise to deliver defined
functionality at a specific level of quality by a
certain date. A commitment can be the same as the
estimate, or it can be more aggressive or more
conservative than the estimate

7. Distinguish between estimates, targets, and commitments.
7

8. What is an estimation
8
Estimate, Target and Commitments
Good estimation

9. Good estimation
9
A good estimation approach should provide
estimates that are within 25% of the actual results
75% of the time.
However, accurate estimation results cannot be
accomplished through estimation practices alone.
They must also be supported by effective project
control.

10. Estimation and Project Control
10

11. Estimation Track Record
11

12. Project Outcomes by Project Size
12

13. Estimation and Project Control
13
The criteria for a quot;goodquot; estimate cannot be based
on its predictive capability, which is impossible to
assess, but on the estimate's ability to support
project success

14. Reasons to make incorrect
14
estimation

15. Source of estimation Uncertainly
15

16. Cone of Uncertainly
16

17. The cone doesn’t narrow itself
17
Don't assume that the Cone of Uncertainty will narrow itself. You must force the Cone
to narrow by removing sources of variability from your project.

18. The cone doesn’t narrow itself
18
Estimation Error by Software-Development Activity

19. The cone doesn’t narrow itself
19
Relationship Between the Cone of Uncertainty and
Commitment
Meaningful commitments are not possible in the early,
wide part of the Cone. Effective organizations delay
their commitments until they have done the work to
force the Cone to narrow.
Meaningful commitments in the early-middle part of the
project (about 30% of the way in) are possible and
appropriate.

20. Chaotic Development Process
20

21. Common examples of project
chaotic
21
Requirements that weren't investigated very well
in the first place
Lack of end-user involvement in requirements
validation
Poor designs that lead to numerous errors in the
code
Poor coding practices that give rise to extensive
bug fixing
Inexperienced personnel
Incomplete or unskilled project planning
Abandoning planning under pressure
Developer gold-plating

22. Unstable requirements
22

23. Unstable requirements
23
If requirements cannot be stabilized, the Cone of
Uncertainty can't be narrowed, and estimation
variability will remain high through the end of the
project.
To deal with unstable requirements,
consider project control strategies
instead of or in addition to estimation
strategies.

24. Omitted activities
24

25. Software-Development Activities
Commonly Missing
25
Ramp-up time for new team members
Mentoring of new team members
Management coordination/manager meetings
Requirements clarifications
Maintaining the revision control system
Review of technical documentation
Coordination with team members

26. Software-Development Activities
Commonly Missing
26
Technical support of existing systems during the
project
Maintenance work on previous systems during the
project
Integration work
Reviewing plans, estimates, architecture, detailed
designs, stage plans, code, test cases, and so on
Input to user documentation and review of user
documentation

27. Non-Software-Development
Activities Commonly Missing
27
Vacations and Holidays
Sick days
Company/Department meetings
Hardware and Software problems
Setting up new Workstations

28. Unfounded optimism
28

29. Common optimistic issues
29
We'll be more productive on this project than we
were on the last project.
A lot of things went wrong on the last project. Not
so many things will go wrong on this project.
We started the project slowly and were climbing a
steep learning curve. We learned a lot of lessons
the hard way, but all the lessons we learned will
allow us to finish the project much faster than we
started it.

30. Common optimistic issues
30
We will recruit/staff the talents or experience to
team and timeline could be reduced
We have experience with that kind of project
before

31. Subjectivity and Bias
31

32. Off-The-Cuff Estimates
32

33. Avoid Off-The-Cuff Estimation
33
Average error of estimates in these 24 groups of estimators for
off-the-cuff estimates versus estimates that go through a group
review process.

34. Unwarranted Precision
34
VS
Accuracy Precision
Accuracy refers to how close to the real value a number is. Precision refers merely to how
exact a number is. In software estimation, this amounts to how many significant digits an
estimate has. A measurement can be precise without being accurate, and it can be accurate
without being precise.

35. Unwarranted Precision
35
VS
High accuracy but low precision High precision but low accuracy
Example Comments
This project will take 395.7 days, ± 2 months Highly precise, but not accurate to the
precision stated
This project will take 1 year Not very precise, but could be accurate
This project will require 7,214 staff hours Highly precise, but probably not accurate to
the precision stated
This project will require 4 staff years Not very precise, but could be accurate

36. Other sources of error
36
Unfamiliar business area
Unfamiliar technology area
Incorrect conversion from estimated time to project
time (for example, assuming the project team will
focus on the project eight hours per day, five days
per week)
Misunderstanding of statistical concepts (especially
adding together a set of quot;best casequot; estimates or a
set of quot;worst casequot; estimates)
Budgeting processes that undermine effective
estimation (especially those that require final
budget approval in the wide part of the Cone of
Uncertainty)

37. Other sources of error (cont.)
37
Having an accurate size estimate, but introducing
errors when converting the size estimate to an effort
estimate
Having accurate size and effort estimates, but
introducing errors when converting those to a
schedule estimate
Overstated savings from new development tools or
methods
Simplification of the estimate as it's reported up
layers of management, fed into the budgeting
process, and so on

38. Estimate Influences
38

39. Determinate factors to estimate
39
Project size
Don't assume that effort scales up linearly as project
size does. Effort scales up exponentially.
The kind of software
Personnel factors
The programming language

40. Fundamental Estimation Techniques
40

41. Choosing Estimation Techniques
41
Project size
Software Development Style
Development Stage
Accuracy Possible

42. Fundamental Estimation Techniques
42
Count, Compute, Judge
•
Calibration and Historical Data
•

43. Applicability
43
Count Compute
What's estimated Size, Features Size, Effort, Schedule, Features
Size of project SML SML
Development Stage Early-Late Early-Middle
Iterative or sequential Both Both
Accuracy possible High High

44. Count First
44
If you can count the answer directly, you should do
that first. That approach produced the most
accurate answer in the story.
If you can't count the answer directly, you should
count something else and then compute the answer
by using some sort of calibration data
Count if at all possible. Compute when you
can't count. Use judgment alone only as a last
resort.

45. What to count
45
Find something to count that's highly correlated with
the size of the software you're estimating
Find something to count that's available sooner rather
than later in the development cycle
Understand what you're counting
Find something you can count with minimal effort

46. Convert Counts to Estimates
Quantity to Count Historical Data Needed to Convert the Count to an Estimate
Marketing requirements • Average effort hours per requirement for development
• Average effort hours per requirement for independent testing
• Average effort hours per requirement for documentation
• Average effort hours per requirement to create engineering requirements from marketing
requirements
Features • Average effort hours per feature for development and/or testing
Use cases • Average total effort hours per use case
• Average number of use cases that can be delivered in a particular amount of calendar time
Stories • Average total effort hours per story
• Average number of stories that can be delivered in a particular amount of calendar time
Change requests • Average development/test/documentation effort per change request (depending on
variability of the change requests, the data might be decomposed into average effort per
small, medium, and large change request)
Function Points • Average development/test/documentation effort per Function Point
• Average lines of code in the target language per Function Point
Defects found • Average effort hours per defect to fix
46
• Average effort hours per defect to regression test
• Average number of defects that can be corrected in a particular amount of calendar time
Test cases already • Average amount of release-stage effort per test case
written

47. Use Judgment Only as a Last Resort
47
Expert judgment is the least accurate means of
estimation. Estimates seem to be the most accurate
if they can be tied to something concrete

48. Fundamental Estimation Techniques
48
Count, Compute, Judge
•
Calibration and Historical Data
•

49. Applicability
49
Calibration with Calibration with Calibration with Project-Specific
Industry-Average Data Organizational Data Data
What's Size, Effort, Schedule, Size, Effort, Schedule, Features Size, Effort, Schedule, Features
estimated Features
Size of project SML SML SML
Development Early-Middle Early-Middle Middle-Late
Stage
Iterative or Both Both Both
sequential
Accuracy Low-Medium Medium-High High
possible

50. Data to collect
50
Size (lines of code or something else you can count
after the software has been released)
Effort (staff months)
Time (calendar months)
Defects (classified by severity)

51. Improved Accuracy and Other
Benefits of Historical Data
51
Accounts for Organizational Influences
How complex is the software, how much documentation
is required, how precedented is the application
Is the project manager free to remove a problem team
member from the project, or do the organization's
Human Resources policies make it difficult or impossible
to remove a problem employee?
Is the team free to concentrate on the current project, or
are team members frequently interrupted with calls to
support production releases of previous projects?

52. Improved Accuracy and Other
Benefits of Historical Data (Cont.)
52
Can the organization add team members to the new
project as planned, or does it refuse to pull people off
other projects before those projects have been
completed?
Does the organization support the use of effective
design, construction, quality assurance, and testing
practices?
Can the project manager depend on team members
staying until the project is complete, or does the
organization have high turnover?

53. Improved Accuracy and Other
Benefits of Historical Data (Cont.)
53
Avoids Subjectivity and Unfounded Optimism
Use historical data as the basis for your productivity
assumptions. Unlike mutual fund disclosures, your
organization's past performance really is your best
indicator of future performance.
Reduces Estimation Politics
Use historical data to help avoid politically charged
estimation discussions arising from assumptions like quot;My
team is below average.quot;

54. How to calibrate
54
The ultimate goal of collecting data is to convert the
data to a model that you can use for estimation.
Example:
Our developers average X lines of code per staff month.
Our team is averaging X staff hours per use case to create
the use case, and Y hours per use case to construct and
deliver the use case.
Our testers create test cases at a rate of X hours per test
case.
In our environment, we average X lines of code per function
point in C# and Y lines of code per function point in Python.
On this project so far, defect correction work has averaged
X hours per defect.

55. Using Project Data to Refine Your
Estimates
55
Even if you don't have historical data from past
projects, you can collect data from your current
project and use that as a basis for estimating the
remainder of your project.
Your goal should be to switch from using
organizational data or industry-average data to
project data as soon as you can. The more iterative
your project is, the sooner you'll be able to do this.

56. Estimation process of Agile Mobile
56

57. What is story point?
57
Story point is a random measure used by Scrum teams. This
is used to measure the effort required to implement a story.
Story points do give some indication of how much time was
spent in a sprint.
Example:
At the end of a 10 day sprint a team of 4 covers 40 story
points.
10 days = 10 * 8 working hours = 80 hours. = 320 total
hours as there are 4 developers. That equated to 160
pairing hours.
160 divided by 40 = 4 hours pair hours per story point.

58. Agile Mobile estimation method
58
We use the story point as the unit to measure the
size of features
Maintain the project/product historical data
(size/effort/schedule/defects) to improve the
estimate precision

59. Politics, Negotiation, and Problem Solving
59

60. Attributes of Executives
60
1 Executives will always ask for what they want.
2 Executives will always probe to get what they want if they don't get it initially.
3 Executives will tend to probe until they discover your point of discomfort.
4 Executives won't always know what's possible, but they will know what would be good for the business if it were possible.
5 Executives will be assertive. That's how they got to be executives in the first place.
6 Executives will respect you when you are being assertive. In fact, they assume you will be assertive if you need to be.
7 Executives want you to operate with the organization's best interests at heart.
8 Executives will want to explore lots of variations to maximize business value.
9 Executives know things about the business, the market, and the company that you don't know, and they may prioritize your
project's goals differently than you would.
10 Executives will always want visibility and commitment early (which would indeed have great business value, if it were
possible).

61. Political Influences on Estimates
61
External Constraints
Be aware of external influences on the target.
Communicate that you understand the business
requirements and their importance.
Negotiating an Estimate vs. Negotiating a
Commitment
You can negotiate the commitment, but don't negotiate
the estimate.

62. Political Influences on Estimates
62
What to Do if Your Estimate Doesn't Get Accepted
Let it be
Responsibility of Technical Staff to Educate
Nontechnical Stakeholders
Educate nontechnical stakeholders about effective
software estimation practices.

63. Problem Solving and Principled
Negotiation
63
Treat estimation discussions as problem solving, not
negotiation.
Recognize that all project stakeholders are on the
same side of the table.
Everyone wins, or everyone loses.

64. References
64

65. References
65
Steve McConnell, Software Estimation: Demystifying
the Black Art – Microsoft Press
Mike Cohn, Agile Estimating and Planning – Prentice
Hall