With Companies like Google and Spotify, not caring about deadlines, why should we care about estimating software?
Why should we estimate? Estimation is not just about meeting deadlines. Managers need to know if the cost of building is worth the effort. Banks, budgets and backers need to gage how their investment is going to be used.
Why should developers Estimate? Developers own the code Magne Jørgensen + Stein Grimstad proved If you have any inkling of budget or time line, Your estimate will be biased. Project Managers and Owners know this information and might try to “fit” the cost into what they are willing to spend
What Developers do Wrong?
Developers are capable of taking problems and break it down into smaller parts. When it comes to Estimating, we follow arbitrary means that can cause projects to go over budget or worse, fail completely.
A story about estimation that went “well”
I needed a developer that was working on another project. He was working with someone who has been programming 30 years. Both were not sure on how long it would take to complete the huge laundry list of items. I sat down with both of them, went over the whole project and broke it up into smaller parts. We then came up with estimates for each items, then sat down with the Project Manager to come up with a time line the client was happy with. The project goes along, for a month, then panic. One of the libs we were using was not working with some version of Safari. Everyone panicked to get a fix in. There was fear that we would miss the deadline. Which would set back my project. We got a fix in, deployed all the changes, and when the smoke cleared, we found that we launched a day early.
How did we do it?
When dealing with estimates, you are fighting a battle of uncertainty.
Its easy to estimate what you know Its hard to estimate what you don’t know Its very hard to estimate what you don’t know you don’t know
**Use driving to work example**
Requirements – Wordy Expressions
Managers try to “sell” requirements to you. Which means that some times they will add “fluff” to requirements. Wordy expressions add useless information to a statement
Requirements – Misplaced Modifiers
Misplaced modifier happens when a word, phrase or clause is improperly separated from the word it describes. In this case, we have valid next to first and last name. Valid modifies First and Last name. Does that mean that we can enter an invalid email address or phone number?
Groucho Marx has a famous one: I shot an elephant in my pajamas the other day. How it got into my pajamas Ill never know
Requirements – Clarity
Now that we have cleaned up the format and grammar of the requirement, we now look for smells in the requirement. For example, what is considered a “valid” phone number?
Requirement – Smells
If estimation is a battle for uncertainty, requirement gathering is a battle against ambiguity. Words used in requirements that cause ambiguity, I like to call Requirement smells. Like Code Smells, these words or phrases raise some red flags about the requirements. Take the time to clarify the requirements before making an estimate.
Requirements – Finial
Once you have defined the requirements as best you can, you can now start estimating. We now see that our one requirement for a contact form has grown into a much bigger project then we thought. We need to break down our requirements into smaller parts. Break the requirement into smaller more manageable units. We can then take a look and see which parts have the most uncertainty.
Tools for Estimating Before getting into some techniques, we are going to need some more information. Using the following tools we can then break down our requirement
Historical data Take current data about development and proxy it to new requirements. Start tracking metrics like LOC, Number of Functions, Avg LOC / Function. Apply time to each of those metrics to get a rough idea on how long it takes to create each metric: LOC / Hour / Day.
I wrote a script that would run git commit every hour to help with this
Dry run / Unit test You don’t need to use a full stack testing framework, but you can test out some critical functions. If you have not previous experience with a service, you therefore have no historical data to base your estimate on. Spending an hour or two on testing out the logic, can provide you with better insight on the complexity for the requirement.
Even if you have worked on something similar in the past, do a dry run for the more complex tasks. I was asked to connect to the OH tax service using a SOAP service. In the past I made many SOAP calls, so my estimate reflected that experience. After spending about 15 hours of my 12 hour estimate, I was unable to make the connection for technical reasons and the requirement was dropped (I was told by the developer that I need to use .NET or Java and not PHP in order to use the service).
Confidence Interval (CI) This is a statistical model that represents uncertainty. It is calculated by using means and variances. We see them in the real world with hurricane paths. They are great because we do not need to “pad” our estimate. The interval uses a High and Low range that represents our 90% confidence that the “True” value is between them.
We are going to do a practice calibration test. Three questions have a number value, for these try use the confidence interval. Three questions will be true or false, do not answer those with null ;).
Take a lunch break come back in an hour and try your estimate again. Clear your mind and try the estimate again. Don’t read your 1st estimate before trying again to avoid anchoring to your original estimate.
Pros and Cons
Make a list of things that will happen if your estimate is right and if your estimate is wrong. This helps bring clarity to the problem and remove some bias. After the list, try again
Narrow down your range by using absurd values for your CI and making them smaller. For example, for the wingspan of a 747, starting with a range of 1 to 1000 ft. is absurd. Is 80 to 250 ft sound better? What about 180 to 220 ft?
This works on a gut feeling. Imagine a spinner that pays out 90% of the time. You choose between your estimate and betting against the spinner. If you choose the spinner, you most likely not confident your value. If you choose your estimate, you might be overconfident and your range is too wide.
A simple estimation tool to get an idea on effort. Classify features into Very Small, Small, Medium Large, Very Large. You then have an idea on how much work is needed based on Historical data or Industry average. Keep track of your estimate with hours.
Wideband Delphi AKA Group
Based on the statistical Law of Large Numbers. Where by the more information you have, the closer your average is to the true value. This requires a team of at lease 4 people and works best with about 10 total. First choose a coordinator. The coordinator presents the feature requested and takes estimates from each member, and averages the numbers. The coordinator then presents the data to the team. A vote is then cast and if it fails, the team estimates again. It is critical that estimates are kept secret to avoid bias. Traditional Wideband Delphi fails to represent uncertainty since the average is voted on, I recommend that you take ranges and then average the high and low numbers.
As stated in the beginning of this presentation, we are going to avoid math as much as possible. However you cannot talk about probabilities with out talking about Bayes Theorem. It was developed by Howard Bayes in the 17th century and is widely used in many applications today from predictive text to suggested ads. The simplicity of the equation makes for highly complex “Bayesian trees”. The take away from Bayes Theorem is this: when you have more information about a probability, you must change your original estimate. The estimate at the start of a project might not always reflect the estimate half way through. By that point you have more clarity and now can better predict the outcome.
Once all the features have estimates attached, how do you set the priorities on when items are going to get done? Most of the time we use order them in High, Medium, or Low. How effective is that? Based on surveys from project managers, you will find that ~85% of your tasks become High, ~10% Medium and ~5% Low. Three level scale is an easy way to avoid this trap. We create a matrix of Importance vs Urgency. By comparing the values, you get a grasp of the priority scale.
Priorities Prioritization Spreadsheet
This is the ultimate way to reduce bias.
Have the customer rate relative benefit for each feature on a scale of 1-9. 9 is extremely valuable.
Have the owners Estimate the penalty on a scale of 1 – 9, 9 means a serious impact on business
Developers Rate the relative on a scale of 1 – 9, 1 is quick an easy. Then rate the technical risk. 1 means you can do in your sleep
Then sort the list descending on priority. The priority is based on the Value % /
Dealing with the rest of the company can be a challenge. Keep in mind that there will be politics everywhere you go but you can curve some of resistance you will get. When dealing with managers or product owners remember this: You are responsible for the code.
Imagine a scenario where a patient needs to be operated on by a Dr. The patient is on the table and sees the Dr. washing his hands. If the patient yells out to the Dr. to stop washing his hands and get in the OR to start, is the Dr. going to listen? The patient is the boss because he is paying the Dr. but the Dr. is going to act in a manner that benefits the Patient.
Remove people from the problem
Everyone will want everything done yesterday. You also have your own needs for the project that must be met. If you focus on what is best for the project (or better yet the dollar amount), it helps change the perspective. In the example early with connecting to the FooBar Rest service, if development will cost $2,000 plus another $500 per month to maintain, but saves $2.00 per lead. If you only get 2 leads a day, the savings per lead per month is only about $120. If a sales rep is demanding that this feature be implemented, prove that there is a loss per month with this service, make the priority lower.
Focus on Interests not Positions
If the estimate takes the project longer than the ship date, work out with the project owners what you can deliver in the time. Try to get features implemented with known bugs and workarounds that you can fix the ship. If this is needed for a trade show, work out a “white page” demoing the features that you wont be able to deliver
Hours scale more in a day then days do in a week.
It takes 9 months to make a baby, 9 women cannot make a baby in one month. Developers need to be trained, and oriented to a project.
Iterate your estimates in the same way that you iterate code. When you notice a job stopper, inform the party early on in the cycle rather than the end. This will help keep budgets in line and expectations high.
Why even estimate?
Why should developers estimate?
I'll tell you how I estimate things. I sit down and figure out how long I think it would
take me to do it. Then I double that time and then I push it up to the next point on
the “Time Progression Scale”. Were the “Time Progress Scale” starts off: seconds,
minuets, hours and days. So if I think something is going to take me 4 hours to do.
I will double it to 8 and then I would say it would take me 8 days to get it all
completely done start to finish. And I'm usually right, because there are so many
impediments. If you are the only one doing it yourself your schedule can be very
accurate. Once you have to involve other people, you might as well just make
numbers up. You might as well say “I think this will take me Blue days to get done”
Requirements are the key
Measure what is measurable, but make measurable what is not so
Our new contact form for the sales team to call or email potential leads, will require
the potential leads to provide the following contact information: First and Last name,
Email address, and Phone number
The contact form has fields for entering a valid First and Last Name, Email
address and Phone number
Top Tip: Prevent children from ingesting dangerous medicines by locking
them in a childproof cupboard. 3 children per cupboard is a good fit
-Periwinkle Jones @peachesanscream
The contact form has fields for First and Last Name, Email address and
Phone number. All fields are required and must be validated
+44 871 984 6352
+852 2280 2898
+91 11 2679 1234
Valid Phone numbers:
fast, rapid, efficient Use a set time: "5 seconds"
valid, including but not limited too,
etc., and so on
Describe what is valid or invalid. use
maximize, minimize, optimize, at
least, between, several
Be sure to include appropriate values
simple, easy, quick, user-friendly Describe what makes it these
reasonable, when necessary How do you make this judgment?
Source: Software Requirements 2 – Karl Weigers
In order to estimate, you must define the parameters of what you are
First and Last name must be between 3 and 100 characters
Email complies with RFC 822 and no longer than 300 characters
Phone Number < 25 characters and validated with Foo-Bar REST service
Dry run / Unit test
Confidence Interval (CI)
It is better to be roughly right than precisely wrong.
- John Maynard Keynes
What is the wingspan of a 747
How far is NY from LA
The average house in the United States uses how many gallons/liters of water
Francis Scott Key wrote the lyrics, but not the music, for the American National
28 degrees Fahrenheit is colder than -15 degrees Celsius.
America On Line purchased Netscape.
Pros and Cons
Q. What is the wingspan of a 747
A. 211 ft (64m)
Q. How far is NY from LA
A. 2,808 mi (4,519 km)
Q. The average house in the United States uses how many gallons/liters of
water per day?
A. 350 g (1,324 l)
Q. Francis Scott Key wrote the lyrics, but not the music, for the American
Q. 28 degrees Fahrenheit is colder than -15 degrees Celsius.
Q. America On Line purchased Netscape.
Fuzzy Logic / Tee Shirt
Size Average LOC
Very Small 127
Very Large 1,988
Source: Software Estimation 2 – Steve McConnell
Chuck Reeves @manchuck
Software Estimation: Demystifying the Black Art – by Steve McConnell |
Software Requirements: Practical Techniques for Gathering and Managing
Requirements – by Karl Wiegers
How to Measure Anything: Finding the Value of Intangibles in Business –
by Douglas Hubbard | ISBN-13: 978-1118539279
Dev Hell Podcast - Episode 29: Snappy Answers to Stupid Questions