-
1.
ESTIMATION
PROTIPS
ATLANTA PHP USER GROUP
SEPTEMBER 2013
-
2.
ABOUT ME
• Director of
Development
• 5+ years of full-time
web development
• Learned estimation
from the school of
hard knocks
-
3.
YOUR ANSWER?
-
4.
ESTIMATES ARE
NOT PROMISES
PROTIP #1:
-
5.
“GOOD” ESTIMATES
A good estimation approach should
provide estimates that are within 25%
of the actual results 75% of the time.
Conte, Dunsmore, and Shen (1986)
-
6.
SEEKING CERTAINTY
Sadly, people asking for control or
visibility really want certainty.
Which doesn’t exist.
Dan North
https://twitter.com/tastapod/status/116271851767992320
-
7.
DISTINCTIONS
Target: a stated desirable business
objective
Commitment: a promise to deliver a
specific product within a timeframe
-
8.
DEFINITION
A good estimate is an estimate that
provides a clear enough view of the
project reality to allow the project
leadership to make good decisions
about how to control the project to hit
targets.
Steve McConnell, Software Estimation
-
9.
GUTS LIE
PROTIP #2:
-
10.
380,000 LBS
HEAVIEST BLUE WHALE EVER RECORDED
-
11.
REALISTIC?
-
12.
BOOK
RECOMMENDATION
-
13.
HOFSTADTER’S LAW
“It always takes longer than you
expect, even when you take into
account Hofstadter’s Law.”
Douglas Hofstadter
Gödel, Escher, Bach: An Eternal Golden Braid
-
14.
TIME FRAMES
“With software estimation you've only
realistically got a choice of 5 mins, 1
hour, 1-2 days, about a week, and then
all bets are off.”
Rob Bowley
https://twitter.com/robbowley/status/115430969825181696
-
15.
WHY ARE ESTIMATES
SO HARD?
-
16.
IT’LL BE DIFFERENT
THIS TIME!
-
17.
THE PLANNING FALLACY
Students estimated their senior thesis
completion time in a 1994 study:
27.4
33.9
48.6
55.5
0
10
20
30
40
50
60
Best Case
Expected Case
Worst Case
Actual
Source: Wikipedia
-
18.
PREMATURE
ESTIMATION IS
SABOTAGE
PROTIP #3:
-
19.
DON’T ESTIMATE
If there’s as much chance of you
coming up with something meaningful
by rolling some dice or rubbing the
estimate goat then what purpose are
you satisfying by doing so?
Rob Bowley
-
20.
CONE OF UNCERTAINTY
-
21.
OVERESTIMATION
• Inflated prices – might lose the job
• Lack of urgency – project time fills
up the estimate when it could have
been done faster
• Procrastination
-
22.
UNDERESTIMATION
• Inadequate planning
• Missed deadlines
• Overwork, burnout
• More bugs
• Technical debt
• Damage control
• Unplanned interim releases
• Meetings proliferate
-
23.
BIG TEAMS ARE
SLOWER THAN
SMALL ONES
PROTIP #4:
-
24.
TIME
=
ESTIMATE
÷
AVAILABILITY
-
25.
TEAM EFFICIENCY
Developers Communication
Paths
Individual
Efficiency
Team
Capacity
1 0 100% 1x
2 3 75% 1.5x
3 6 67% 2x
4 10 63% 2.5x
5 15 60% 3x
6 21 58% 3.5x
7 28 57% 4x
8 36 56% 4.5x
9 45 56% 5x
10 55 55% 5.5x
Source: Paul M. Jones, http://paul-m-jones.com/archives/1591
-
26.
BEWARE
UNWARRANTED
PRECISION
PROTIP #5:
-
27.
“533.5 hours”
vs
“13 days”
vs
“3 weeks”
-
28.
COUNT ALL THE
THINGS
PROTIP #6:
-
29.
TIME FRAMES
“With software estimation you've only
realistically got a choice of 5 mins, 1
hour, 1-2 days, about a week, and then
all bets are off.”
Rob Bowley
https://twitter.com/robbowley/status/115430969825181696
-
30.
DECOMPOSITION AND
RECOMPOSITION
1. List all the features
2. Break the features into sub-
features
3. Break the sub-features into
components
4. Estimate the components
5. Add the estimates up
-
31.
LAW OF LARGE NUMBERS
The tendency for errors on the high
side and errors on the low side to
cancel each other out.
i.e.,
The accuracy of the sum is greater
than the accuracy of the individual
estimates.
-
32.
PAUL JONES’ METHOD
1. List all the controllers required for
each feature
2. List all the methods required for
each controller
3. Estimate 1 dev-pair day per
controller method
-
33.
BRANDMOVERS
METHOD
1. List all the logical features required
2. Break down each feature into small
logical components
3. List all the pages and modals required for
each feature
4. Estimate the back-end time required for
each logical component
5. Estimate the front-end time required for
each page
6. Sum up the back-end and front-end totals
-
34.
WHEN IN A
PINCH, USE A
PROXY
PROTIP #7:
-
35.
PROXY ESTIMATION
1. Assign a size classification to each
feature
2. Compute the average time required
for similarly-sized features from
actual past projects
3. Create estimate ranges for each
feature based on past performance
4. Sum the result
-
36.
PROS
• Easier
• Faster
-
37.
CONS
• Less accurate
• Requires collection and archival of
project historical data on a per-
feature basis
-
38.
STORY POINTS
• Uses a point scale: 1, 2, 4, 8, 16
• Break down the project into epics and
stories
• Assign a point value to each story
• Schedule releases at regular intervals
• The number of points completed per
release is known as “velocity”
• Use the velocity to plan and estimate
the delivery dates for future releases
-
39.
EXAMPLE
Iteration 1
• 27 story points delivered
• 12 staff weeks expended over 3 calendar weeks
• Effort = 27 points / 12 weeks = 2.25 points/week
• Schedule = 27 points / 3 weeks = 9 points/week
Iteration 2 projection
• 33 story points scheduled
• Effort = 33 points / 2.25 points/week = 15 staff weeks
• Schedule = 33 points / 9 points/week = 4 calendar weeks
-
40.
T-SHIRT SIZING
• Assign a T-shirt size for development cost
• Assign a T-shirt size for business value
• Create a table of business value to
development cost ratios
• Look up the net business value for each
feature based on the dev cost and
business value T-shirt sizes
• Prioritize the features in order of net
business value
-
41.
EXAMPLE
Feature Business Value Dev Cost
Feature A L S
Feature B S L
Feature C L L
Feature D M M
Feature E M L
-
42.
VALUE TO COST RATIOS
XL L M S
XL 0 4 6 7
L -4 0 2 3
M -6 -2 0 1
S -7 -3 -1 0
Development Cost
BusinessValue
-
43.
BIZ VALUE EXAMPLE
Feature Business
Value
Dev Cost Net Value
Feature A L S 3
Feature C L L 0
Feature D M M 0
Feature E M L -2
Feature B S L -3
-
44.
YOU CAN’T
NEGOTIATE MATH
PROTIP #8:
-
45.
PROBLEM SOLVING
When the estimate and target conflict:
• Negotiate features
• Negotiate time
• Negotiate price
-
46.
ATTITUDE
• Try to be helpful, offer solutions
• Be creative
• Examine what can be done in
parallel to save time
• Be firm – you can’t change the laws
of physics
-
47.
QUESTIONS?
-
48.
A HORROR STORY
ONE FINAL WORD
-
49.
THE SETTING
• Former employer of mine
• Start-up, naïve and inexperienced
• Needed cash bad
-
50.
THE CLIENT
• Local company in Atlanta
• Had four separate systems in place
for managing customer data, billing,
inventory, and fulfillment
• Wanted this unified and streamlined
into a web-based backoffice
application
• Wanted a customer-facing portal for
online ordering and bill paying
-
51.
THE ESTIMATE
• Estimated at 1,039 man-hours
• Normal hourly rate was $120/hr
• We did a fixed-bid for $50k, at an
effective hourly rate of $48/hr
-
52.
THE FALLOUT
• 18 months later…
• 2,500 man-hours
• 1,500+ Subversion commits
• Lots of “unknown
unknowns”, hidden
complexities, and scope creep
-
53.
THE MORAL
• Don’t succumb to pressure to be
optimistic when estimating
• Use a good estimation methodology
• Try not to do fixed bidding
• Always have a thorough scope
before starting
-
54.
THANK YOU!
• http://brandmovers.com
• http://jonathonhill.net
• @compwright
Hello! I’m Jonathon Hill, been in the industry full time for
Brandmovers is a digital brand marketing companyOur clients are household names – names like Disney, Kohl’s, and Dr PepperProjects vary from small to very largeCurrently hiring a mid-level PHP and front-end developer
This is you. You’re peacefully composing code into a profoundly elegant masterpiece, or maybe you’re just staring at your desk, when…
suddenly this guy shows up. He’s got this big idea that he’s pitching to some VCs.You get the30 secondelevator pitch, and then he says,“I’m gonna need you to go ahead and give me an estimate on that so I can set a budget, and be sure to go ahead and use the new coversheet, mmkay?”
Your answer?How you answer this question is extremely important.
Judging by the number of people that think that “estimate” and “quote” are synonymous, I think there is a lot of misunderstanding on what an estimate actually is.There is nothing at all binding in an estimate, it is more like a measurement.
An estimate is simply a “tentative evaluation or rough calculation”.Even a good estimate (by the textbook definition) has a 25% chance of falling short.Would you buy a car that had a 1-in-4 chance of not starting when you try to crank it?If even the best estimation process has a 25% uncertainty, and you allow estimates to be confused for commitments, then from management’s perspective, or the client’s perspective, you’re just like that car – unreliable.
Dan North sez…
Targets:“We need v2.1 ready to demo at a trade show in May.”“Our budget is $2mm, we must limit the cost of the next release to that budget.”
Here’s the problem: estimates and targets often collide. Good estimates are probability statements based on what is known about the project and available resources, and targets are specific business objects that are sometimes quite solid.At Brandmovers, if we’re building a promotion to help generate buzz and excitement for the next Avengers movie, and that movie launches on a particular date, it isn’t really an option to delay launching the promotion if the project runs over.So if an estimate isn’t a promise, what is a good estimate?
To illustrate the point, I’d like to do a little exercise.
I’d like you to estimate the weight, in pounds, of the heaviest blue whale ever recorded.No cheating, don’t Google it.Give a high and a low estimate.Here’s the catch: I want you to estimate this with a wide enough range so that you are 90% confident that the correct answer is in the range that you set.You’ve got one minute. Go.
Now, I’d like to get a show of hands of how many of you captured the correct answer in your range.Almost everyone gets this wrong.
You might be thinking, “c’mon, this is not a realistic exercise.”Folks, estimating in the face of this kind of uncertainty is business as usual for software estimators. Fortunately, there are techniques that will help you get within 25% of the actual number 75% of the time.The other takeaway here is unless you are into statistics and you have have a mathematical reason for saying so, don’t use terms like “90% confident” or “75% confident.”
Before we go any further, I’d like to share a book recommendation.The exercise we just did comes from chapter 2 of the bookSoftware Estimation, by Steve McConnell.Steve is a former Microsoft employee and also authored the book Code Complete.Software Estimation is pretty much required reading for anyone who is responsible for estimating software projects. This book really helped me to understand why good estimation practices work, and what I had been doing wrong for years before.
Hofstadter’s Law illustrates why estimation is so hard, and why so many people get it wrong.In fact, the difficulties with producing even a ballpark estimate and the the tendency to misunderstand and abuse estimation have led some to seek workflows which eliminate estimation altogether. Some agile methodologies do this.Sadly, the dynamics and business models of many companies do not allow such a luxury.
Rob Bowleysez…
Optimism bias is when you believe you are safer from a negative event than others.It’s the idea that you can “beat the odds” or it “won’t happen here because” you are somehow immune to the problems that others have faced.
Similarly, the planning fallacy says,“It’ll be different this time!”
Only 30% of the students completed their thesis in the time they estimated.The remarkable finding from this study is just how far off the mark these students were in their estimates – the average actual thesis completion time was longer than even the average worst-case estimate.Think about the implications of this on your own projects.What would happen if 3 out of 4 of your projects took longer than even your worst-case estimate?
A “black swan” is a rare, unpredictable, high-impact event that in retrospect seems not so improbable.While it is true that it is hard to account for a black swan event in your estimates, my real point in bringing this up is to talk about how unknowns make it almost impossible sometimes to produce a useful estimate.Secretary of Defense Donald Rumsfeld is famous for saying,“There are known knowns; there are things we know that we know.There are known unknowns; that is to say, there are things that we now know we don't know.But there are also unknown unknowns – there are things we do not know we don't know.”It’s the unknown unknowns that really get you.
Provide an estimate, even a range, when you don’t have enough information is worse than useless, they can actually hurt your business.
Rob goes on to say,“What’s the least worst situation to be in?Having an uncomfortable situation with your boss now or when – based on your estimates – the project’s half a million over budget and 6 months late?”
The cone of uncertainty shows how error-prone early estimates are. The lines of the cone outline the best case accuracy you could have in various stages of the project; the blue cloud of uncertainty shows what the variability actually looks like in practice.The cone doesn’t narrow itself over time, you have to narrow it by making decisions and defining the specifications.Early in the process, few decisions have been made about the project, so the variability in the project time/cost is very wide.It’s similar to asking a builder to give you a cost estimate on building a building without telling him whether it’s a doghouse or a skyscraper, or something in between. If the builder is smart, he’ll ask some questions before putting pen to paper.Avoid estimation prior to completion of the project requirements, and plan to re-estimate as soon as the user interface design is complete.
Typical business concerns about over-estimating a project include things like…
However, the consequences and costs of underestimating are vastly higher than the cost of overestimating.These include…
Let’s say you’re a freelancer, and you get a project. You know everything there is to know about the project, and as such, you have the ability to work at 100% of your capacity on the project.
Calculating the time required to complete the project is a simple matter of dividing your estimate by the amount of time per day or week that you can devote to the project.
One day, you get a project that you’ve estimated out at somewhere between 3-4 months of development time. However, your client has a target completion timeframe of only 6 weeks. What do you do?
Being the smart businessman that you are, you get help. Your buddy is available to help and you offer to split the costs with him.Great! You get get the project done in 1.5-2 months, since you’ve doubled the capacity to work on the project, right?Wrong.It’s because you’ve now introduced communication overhead between the two of you.
It gets worse, the more people you add.
The communication overhead grows at an exponential rate.
The actual numbers may be higher or lower, this is a hypothetical model to illustrate my point.
It is best to use the largest units and the fewest significant digits as possible.Match the way you present your estimate to accurately reflect how probable your estimate is, and to communicate to all project stakeholders that this is an estimate that has a certain amount of variability to it.
We’ve talked about a lot of theory, what estimates are, what they’re good for.Now let’s talk about some practical methodology.
Remember what Rob Bowley said?
This is the underlying idea behind estimation by decomposition.Essentially, you break down the project features, and then you break down those sub-features down even further to the individual components that must be built.Estimate each component, and sum the individual estimates to produce the overall estimate.
There are a few different ways to do this. The first way was pretty hard for me to swallow at first.I learned it from Paul Jones and in practice it works very well.This method includes front-end, back-end, and QA time built in and was created based on his personal observations from working on many different projects.Notice the large unit of measure (1 day), and the fact that it assumes two people will be pair programming.I never understood why this method worked so well until I learned about the law of large numbers.Simple methods are often quite effective! It doesn’t take complex mathematics or estimation modeling software to produce a useful estimate. A spreadsheet and a good method are all you need.
To estimate, use multiples of 4 or powers of 2.Rules of thumb:Small features = 2hMedium features = 4hLarge features = 8hVery large features = 16hAnything larger than 16h needs to be broken down further.For pages:Layouts = 8hPages with different layout = 4hPages with same layout = 2hModals = 2-4hDouble or triple front-end time for responsive design or mobile optimization.
Comes from Agile
Pivotal Tracker calculates velocity and divides the upcoming stories into various releases based on the stories’ point values, your velocity, and the length of your iterations.
This is not a true estimation method, but it is useful when all you need to do is figure out what order to build the features in, and what features don’t make business sense to build.
When targets and estimates collide, what do you do?
I.e. the target is usually negotiable, but an estimate cannot be negotiated.
Attitude is everything, as my granddaddy used to say.