This document discusses common issues with project cost and schedule estimates. It notes that estimates are often biased due to human nature and strategic misrepresentation. Experts providing estimates are likely to be biased as well. Several estimation methods are described along with their advantages and limitations. Historical data from completed projects can help improve estimates by providing an outside view, though data limitations remain. The document advocates using techniques like parametric estimating, reference class forecasting, and adjusting for optimism bias to develop more accurate estimates.
Many people view an estimate as a quick guess that no one believes anyhow. But producing a viable estimate is core to project success as well as ROI determination and other decision making. In decades of studying the art and science of estimating it has become apparent that: most people don’t like to and/or don’t know how to estimate; those that estimate are often always wildly optimistic, full of unintentional bias; strategic misestimating provides misleading estimates when it occurs. However, it is also obvious that viable estimates can make projects successful, outsourcing more cost effective, and help businesses make the most informed decisions.
That is why metrics and models are essential to organizations, providing the tempering with that outside view of reality that is recommended by Daniel Kahneman in his Nobel Prize winning work in estimation bias and strategic mis-estimation. (IT Confidence 2014, Tokyo (Japan))
This presentation show how to perform project risk analysis with uncertainties defined using risk events and event chains.
For more information how to perform schedule risk analysis using RiskyProject software please visit Intaver Institute web site: http://www.intaver.com.
About Intaver Institute.
Intaver Institute Inc. develops project risk management and project risk analysis software. Intaver's flagship product is RiskyProject: project risk management software. RiskyProject integrates with Microsoft Project, Oracle Primavera, other project management software or can run standalone. RiskyProject comes in three configurations: RiskyProject Lite, RiskyProject Professional, and RiskyProject Enterprise.
Maximising Capital Investments - is guesswork eroding your bottomline?Michael McKeon
Globally, organisations waste US$122 million for every US$1 billion invested due to poor project performance. Daniel Galorath, the world’s leading expert in project estimation, explains why - and how to create better outcomes.
An extension on hypothesis testing, this lesson introduces the concepts of a correlation and regression as part of measuring statistical relationships.
Many people view an estimate as a quick guess that no one believes anyhow. But producing a viable estimate is core to project success as well as ROI determination and other decision making. In decades of studying the art and science of estimating it has become apparent that: most people don’t like to and/or don’t know how to estimate; those that estimate are often always wildly optimistic, full of unintentional bias; strategic misestimating provides misleading estimates when it occurs. However, it is also obvious that viable estimates can make projects successful, outsourcing more cost effective, and help businesses make the most informed decisions.
That is why metrics and models are essential to organizations, providing the tempering with that outside view of reality that is recommended by Daniel Kahneman in his Nobel Prize winning work in estimation bias and strategic mis-estimation. (IT Confidence 2014, Tokyo (Japan))
This presentation show how to perform project risk analysis with uncertainties defined using risk events and event chains.
For more information how to perform schedule risk analysis using RiskyProject software please visit Intaver Institute web site: http://www.intaver.com.
About Intaver Institute.
Intaver Institute Inc. develops project risk management and project risk analysis software. Intaver's flagship product is RiskyProject: project risk management software. RiskyProject integrates with Microsoft Project, Oracle Primavera, other project management software or can run standalone. RiskyProject comes in three configurations: RiskyProject Lite, RiskyProject Professional, and RiskyProject Enterprise.
Maximising Capital Investments - is guesswork eroding your bottomline?Michael McKeon
Globally, organisations waste US$122 million for every US$1 billion invested due to poor project performance. Daniel Galorath, the world’s leading expert in project estimation, explains why - and how to create better outcomes.
An extension on hypothesis testing, this lesson introduces the concepts of a correlation and regression as part of measuring statistical relationships.
Mythbusting Software Estimation - By Tood LittleSynerzip
In this webinar, some of the myths that will be explored include:
Historical Estimation Accuracy
Relative Estimation
The Cone of Uncertainty
Velocity
Scope Creep
Wisdom of Crowds
Read more at https://www.synerzip.com/webinar/mythbusting-software-estimation-webinar-october-2014/
GRA - Scenario Planning: Addressing a Capability Gap Affecting Industry Compe...Rebecca Manjra
Exponential population and technology growth is occurring at a rate never before seen in history. Together, these forces have created the data driven world we live in. The business landscape has become more competitive and complex given the increased level of capability required to scale, evolve and rapidly gain market share; shortening the business maturity lifecycle.
A critical success factor to survival and succeed in both nature and business is the ability to learn and implement quickly – to adapt and evolve. By reducing the time it takes for your business to know what’s happening, learn what is needed for success and implement, you can outpace your competitors and capture new opportunities.
Today, there is an imperative to turn the vast seas of data into information, something useable which drives insights and enables us to make decisions which optimally utilise assets and resources. In operational speak, this entire process is enabled by excellence in Scenario Planning.
This presentation covers the relevancy of Scenario Planning today including an analysis of the stages of S&OP maturity as well as a case study with Simplot, a leading Australian food manufacturer and a leader in S&OP maturity and Scenario Planning.
Information Security Risk QuantificationJoel Baese
Overview presentation given at the 8/16/2016 Fayetteville, Arkansas ISACA chapter meeting discussing quantifying risk in the information security field.
What is decision framing in project risk management; identifying problems and assessing situations; defining project objectives; generating alternatives and identifying risks.
For more information how to perform schedule risk analysis using RiskyProject software please visit Intaver Institute web site: http://www.intaver.com.
About Intaver Institute.
Intaver Institute Inc. develops project risk management and project risk analysis software. Intaver's flagship product is RiskyProject: project risk management software. RiskyProject integrates with Microsoft Project, Oracle Primavera, other project management software or can run standalone. RiskyProject comes in three configurations: RiskyProject Lite, RiskyProject Professional, and RiskyProject Enterprise.
Hypothesis Testing: Central Tendency – Normal (Compare 1:1)Matt Hansen
An extension on a series about hypothesis testing, this lesson reviews the 2 Sample T & Paired T tests as central tendency measurements for normal distributions.
Measuring Risk - What Doesn’t Work and What DoesJody Keyser
The topics for this webinar include:
The Problem – Why your method may be a “management placebo” and why that is the biggest risk you have Problems that many methods ignore – and problems some methods introduce What Does Work – Studies reveal some methods show consistent, measurable improvements on the forecasts and decisions of managers
Examples of Real Improvements
Overview of Applied Information Economics (AIE) Process Common Objections to quantitative methods and the misconceptions behind them
Questions & Answers
The Total Economic Impact of NetApp MetroClusterNetApp
In late 2011, Forrester Research started work on a research project commissioned by NetApp that focused on examining the potential return on investment (ROI) enterprises may realize by adopting NetApp’s solution for synchronous data replication known as MetroClusterTM.
Mythbusting Software Estimation - By Tood LittleSynerzip
In this webinar, some of the myths that will be explored include:
Historical Estimation Accuracy
Relative Estimation
The Cone of Uncertainty
Velocity
Scope Creep
Wisdom of Crowds
Read more at https://www.synerzip.com/webinar/mythbusting-software-estimation-webinar-october-2014/
GRA - Scenario Planning: Addressing a Capability Gap Affecting Industry Compe...Rebecca Manjra
Exponential population and technology growth is occurring at a rate never before seen in history. Together, these forces have created the data driven world we live in. The business landscape has become more competitive and complex given the increased level of capability required to scale, evolve and rapidly gain market share; shortening the business maturity lifecycle.
A critical success factor to survival and succeed in both nature and business is the ability to learn and implement quickly – to adapt and evolve. By reducing the time it takes for your business to know what’s happening, learn what is needed for success and implement, you can outpace your competitors and capture new opportunities.
Today, there is an imperative to turn the vast seas of data into information, something useable which drives insights and enables us to make decisions which optimally utilise assets and resources. In operational speak, this entire process is enabled by excellence in Scenario Planning.
This presentation covers the relevancy of Scenario Planning today including an analysis of the stages of S&OP maturity as well as a case study with Simplot, a leading Australian food manufacturer and a leader in S&OP maturity and Scenario Planning.
Information Security Risk QuantificationJoel Baese
Overview presentation given at the 8/16/2016 Fayetteville, Arkansas ISACA chapter meeting discussing quantifying risk in the information security field.
What is decision framing in project risk management; identifying problems and assessing situations; defining project objectives; generating alternatives and identifying risks.
For more information how to perform schedule risk analysis using RiskyProject software please visit Intaver Institute web site: http://www.intaver.com.
About Intaver Institute.
Intaver Institute Inc. develops project risk management and project risk analysis software. Intaver's flagship product is RiskyProject: project risk management software. RiskyProject integrates with Microsoft Project, Oracle Primavera, other project management software or can run standalone. RiskyProject comes in three configurations: RiskyProject Lite, RiskyProject Professional, and RiskyProject Enterprise.
Hypothesis Testing: Central Tendency – Normal (Compare 1:1)Matt Hansen
An extension on a series about hypothesis testing, this lesson reviews the 2 Sample T & Paired T tests as central tendency measurements for normal distributions.
Measuring Risk - What Doesn’t Work and What DoesJody Keyser
The topics for this webinar include:
The Problem – Why your method may be a “management placebo” and why that is the biggest risk you have Problems that many methods ignore – and problems some methods introduce What Does Work – Studies reveal some methods show consistent, measurable improvements on the forecasts and decisions of managers
Examples of Real Improvements
Overview of Applied Information Economics (AIE) Process Common Objections to quantitative methods and the misconceptions behind them
Questions & Answers
The Total Economic Impact of NetApp MetroClusterNetApp
In late 2011, Forrester Research started work on a research project commissioned by NetApp that focused on examining the potential return on investment (ROI) enterprises may realize by adopting NetApp’s solution for synchronous data replication known as MetroClusterTM.
Download Risk Scorecard PowerPoint Presentation Slides to showcase planned methods of risk mitigation. This risk assessment process PowerPoint complete deck includes content ready slides such as risk management lifecycle, types of risks, risk categories, stakeholder’s management and engagement, risk appetite and tolerance, procedure, risk management plan, risk identification, risk register, risk assessment, risk analysis, risk response plan, risk response matrix, risk control matrix, risk items tracking, tools and practices and more. Get your audience focus on risk impact & profitability analysis, risk mitigations strategies, qualitative and quantitative risk analysis, etc. All slides are easy to customize. Users can edit these templates as per their requirements. Furthermore, the scorecard for risk management PPT slides can also be used for related topics such as enterprise risk management, threat management process, project risk management plan and many more. Demonstrate risk evaluation methods using corporate risk scorecard Presentation layout. Avoid inordinate delays with our Risk Scorecard PowerPoint Presentation Slides. Get it done in the allotted duration.
Everyone has been given a 2 paragraph document listing the "scope of services" for a potential project. The client would like an estimate in 48 hours and there are no more details to help you deliver that required fixed bid contract. At the same time, many teams have also been given (or created) a detailed PRD or backlog document and still had a project budget balloon out of control. In this session I would like to discuss the not only the problems associated with estimation and how to avoid them, but more importantly how we can plan for them, turning our estimation process into not only an art, but a science. Well cover how to sell your estimate internally, and arm you with the methodologies to support your numbers. The problem with software estimation The morale The metrics The reality - an estimation metaphor Avoiding Risk Project entry point of sale At what point of the project lifecycle is your first sale? Risk association with point of sale Products in the front, estimations in the back The Elusive Discovery phase How to estimate a discovery How to sell a discovery How to include discovery in a full fixed bid RFP Planning for Risk Estimation types Gut - An art form Comparables - An art/science Factors/formula - A science Contingency Rating systems Formulas Granularity
Estimation - web software development estimation DrupalCon and DrupalCamp pre...Andy Kucharski
Project Estimation Presentation at DrupalCamps. Presentation discussing risks, methods and recommendations when estimating software estimation with a focus on web CMS (drupal)
My presentation at The Richmond Data Science Community (Jan 2018). The slides are slightly different than what I had presented last year at The Data Intelligence Conference.
Crafting a presentation on risk calculator within a limited time is not an easy task. To help you out, we have come up with a professional content ready risk calculator PowerPoint presentation slides. This risk management plan presentation comprises of 25 slides using which you can explain the concept of business risk identification and management in an ideal way. This risk calculator PPT presentation covers slide on a various topic like risk management plan, risk identification, risk analysis, risk impact analysis, risk impact, and probability analysis, qualitative risk analysis, quantitative hazard analysis, and hazard track. This hazard calculator PPT presentation comprises of amazing visuals with thoroughly researched content. Each template is well crafted and designed by our PowerPoint experts. Keeping our consumer needs in mind, we provide additional slides such as meet our team, puzzle, bulb or idea, location, Venn, sticky notes, lego, pie chart, stock chart, and stacked bar to make your PPT task simple. Download our risk calculator presentation slides and impress your audience. Interact closely with the audience through our Risk Calculator PowerPoint Presentation Slides. Be able to establish intimate connections.
With the current expected credit loss (CECL) model for the Allowance on the horizon, bankers will be asked to create future-looking methodologies that adjust for reasonable and supportable forecasts. Without adequate modeling experience, that can be a challenge for community banks and credit unions.
Watch the full webinar here: http://web.sageworks.com/forward-looking-alll-adjustments/
Estimation is dead - long live sizing, by John Coleman 24Nov22.pdfOrderly Disruption
Estimation is dead - long live sizing, by John Coleman 24 Nov 22 to Agile Azerbaijan in person and Pozitive Technologies online
As per https://www.infoq.com/articles/sizing-forecasting-scrum/
The Use of Functional Size in the Industry.pdfNesma
In this webinar, the emphasis is on the use of Functional Size in the Industry, and we focus on several use cases where functional size helps organizations to make impactful decisions based on objective metrics and data.
While traditional performance metrics often measure individual output or adherence to pre-defined plans, measuring performance in agile teams requires a different approach. Agile teams operate in iterative cycles, prioritizing adaptability and learning over rigid goals. So, why do organizations still measure their performance?
By using the right metrics in the right way, organizations can empower their agile teams to thrive and deliver exceptional results.
Software Cost Estimation webinar January 2024.pdfNesma
In this webinar you will learn why Software Cost Estimation is important, what is the Software Cost Estimation Body of Knowledge for Software and the ways you can become a professional certified software cost estimator SCEC!
The journey of UNISON Cost Engineering in the field of automotive software cost estimation started in 2018. The expectation is that in 2030 the cost of software will be 50% of the total production cost of a car. To help the OEM get a proper understanding of the software development cost they need to use some form of size measurement to compare, challenge and control the cost of software development by the software vendors.
The COSMIC battle between David and Goliath - Paul HusseinNesma
No more exhaustive and emotional discussions on price and deliverables. Predictable prices for projects and changes. No escalating maintenance costs. This can only be done by specifying exactly what you want and outsource it to the right service providers that have the required platform already in place.
Succesful Estimating - It's how you tell the story - Amritpal Singh AgarNesma
Estimating the Cost of something is a profession. But then you have to tell the story about the estimate to whoever needs to hear that story. The success of how you tell the story is determining the success of the cost estimate.
CEBoK for Software Past Present Future - Megan JonesNesma
The Cost Estimation Body of Knowledge for Software is in development for a number of years within ICEAA. First as a section of the general CEBoK, but it will be established as a separate CEBoK-S for Software, since software is becoming very prominent within the cost estimation community.
Agile Development and Agile Cost Estimation - A return to basic principles - ...Nesma
Is there a natural tension between agile development and traditional cost management or do we need to return to basic principles? Even when you are flexible, you still need to make a plan, build an estimate and measure what you have achieved.
Resolving Cost Management and Key Pitfalls of Agile Software Development - Da...Nesma
Agile software development does not always live up to the promises. Especially in the field of IT Cost Management. Without proper estimation and tracking the value cannot be made clear.
Project Succes is a Choice - Joop SchefferlieNesma
Project success is a choice. Don't stop thinking about the best way to do a project, agile or not. Select the best competencies to ensure that the project will be successful.
Deze presentatie beschrijft een praktische implementatie van het gebruik van Nesma functiepunten in Agile deliveries. Deze presentatie is gepresenteerd door Richard Sweer van Infinity tijdens de webinar Afrekenen met functiepunten. Voor meer info: www.nesma.org; conference@nesma.org.
A benchmark based approach to determine language verbosity - Hans Kuijpers - ...
Iwsm2014 why cant people estimate (dan galorath)
1. Why Can’t People Estimate:
Estimation Bias and Strategic Mis-
Estimation Webinar2014
Daniel D. Galorath: Founder & CEO
galorath@galorath.com
Copyright Galorath Incorporated 2014
3. ESTIMATION & PLANNING:
An Estimate Defined
• An estimate is the most knowledgeable statement you
can make at a particular point in time regarding:
• Effort / Cost
• Schedule
• Staffing
• Risk
• Reliability
• Estimates more precise with progress
• A WELL FORMED ESTIMATE IS A
DISTRIBUTION
4
6. Human Nature:
Humans Are Optimists
HBR Article explains this Phenomenon:
• Humans seem hardwired to be optimists
• Routinely exaggerate benefits and discount costs
Delusions of Success: How Optimism Undermines
Executives' Decisions (Source: HBR Articles | Dan
Lovallo, Daniel Kahneman | Jul 01, 2003)
7
Solution - Temper with “outside view”:
Past Measurement Results, traditional forecasting, risk
analysis and statistical parametrics can help
Don’t remove optimism, but balance optimism and
realism
13. Josiah Stamp Observation On Data
& Statistics
• “The government [is] extremely fond of amassing
great quantities of statistics. These are raised to the
nth degree, the cube roots are extracted, and the
results are arranged into elaborate and impressive
displays.
• What must be kept ever in mind, however, is that in
every case, the figures are first put down
by a village watchman, and he puts down
anything he … pleases.
• Attributed to Sir Josiah Stamp,
1840-1941, H.M. collector of inland revenue.
Most Data is imperfect…
And much imperfect data is usable
19. ROI Analysis of A New System
Cost of capital 8.0%
Initial Investment Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7
Total
Ownership
Investment $100,000 $100,000
Increase/(dec.) in
revenue ($40,000) $60,000 $110,000 $100,000 $100,000 $150,000 $150,000 $630,000
Increase/(dec.) in
op. exp. $90,000 $70,000 $70,000 $22,000 $24,000 $27,000 $28,000 $331,000
Cash Flow ($100,000) ($130,000) ($10,000) $40,000 $78,000 $76,000 $123,000 $122,000 $199,000
PV of Cash Flow ($100,000) ($120,370) ($8,573) $31,753 $57,332 $51,724 $77,511 $71,186 $60,563
NPV 60,563 $60,563
IRR 13.5% 13.5%
ROI 121% 121.1%
A Complete ROI analysis should analysis risk and
uncertainty as well as likely
20. Manual Estimates: Human
Reasons For Error (Adapted from Goldratt)
• Desire for “credibility” motivates overestimate
behavior (80% probability?)
• So must spend all the time to be “reliable”
• Better approach force 50% probability & have “buffer”
for overruns
• Technical pride causes underestimates
• Buy-in causes underestimates
22
21. Comparison Estimation In A Nutshell
Reference items have known value
Unknown items are to be determined
Gross ratios between reference and unknown item pairings are made
“Slightly
Bigger”
“Much
Smaller”
“Bigger” “Smaller”
Reference item Reference item Unknown item Reference item Unknown item
Estimate Estimate
• Expert Choice most well
known
• SEER Estimate By
Comparison used here
22. SEER Comparison Estimating
Typically Within 10% of Actuals
Decreases in
accuracy are due
to variations in
distributions or #
of reference
items, with no
2
2
2
10
10
100
100
100
45.00%
40.00%
35.00%
30.00%
25.00%
20.00%
15.00%
10.00%
5.00%
0.00%
Accuracy
Max/Min
Ratio
Accuracy for All Ratios, Ref Items, Distributions
75% below min
25% within range
50% within range
75% within range
125% above max
Sorted first by by
max/min ratio and
then accuracy: # of
items, distributions
are not called out
regularity
Notes: 1.
statistical
stress test:
Viable
reference
choices are
most accurate
2. Results from
SEER Estimate
By Comparison
Uses relative +
Monte Carlo
23. Comparative Estimating Can Flag
Inconsistency
What is Consistency?
Not violating transitivity
For A > B > C, saying
“A > B and A > C” is consistent
“A > B and A < C” is inconsistent
Consistency Estimate accuracy
SEER-Estimate By
Comparison assists the
user in identifying
inconsistent comparisons
27. Myth and Reality (Source Hamaker)
• Myth: The more details, the
better the estimate!
• If a 100 WBS element is good…
• Then 1000 elements if better still…
• And 10 elements is to be sneered at…
• And if it is “parametric” it is worse yet
• But on the contrary, if one simply
wants to know what a project
might cost, details are
counterproductive!
• And parametric estimating is the
preferred choice
See next charts
29
for
substantiating
arguments
28. 10/7/
2014
Comparison of Parametric &
Bottoms Up Methods (Source
Hamaker)
Parametric Estimates
Top down
Less detail
Based on performance metrics
Less labor intensive
Quicker
Ease of trade-offs analyses
Parametric database
Not always accepted
“Black Magic” aura
Generally more disciplined
• Standard methodology
• Independent
• Done by trained analysts
• Captures totality of past
programs
Detailed Build-Up Estimates*
Bottoms up
More detail
Based on time and material
Labor intensive
Time consuming
Trade offs need details
Performance standards
Accepted method
Generally understood
More susceptible to distortions
• Optimism/Pessimism
• Special interest/buy-in
• Done by managers/engineers
• Missing
- “I forgots”
- Unknowns
*AKA “labor-material build up”, “grass roots”, “bottoms up”
“engineering estimates”
36. Hubbard: Measure To Reduce
Uncertainty
• Perception that measurement is a point value
is a key reason why many things are
perceived as “immeasurable”
• Measurement: Quantitatively expressed
reduction in uncertainty based on observation
Probability Distribution After Measurement
Probability Distribution Before Measurement
0 0.5 1 1.5 2 2.5 3 3.5 4
Copyright HDR 2010 dwhubbard@hubbardresearch.com
38
Quantity of Interest
38. 40
Gunning for Models (Adapted from Hubbard)
• Be careful of red herring arguments against models
• “We cannot model that…it is too complex.”
• “Models will have error and therefore we should not attempt it.”
• “We don’t have sufficient data to use for a model.”
• “It works but we cant see all data so we should not use it”
• Build on George E. P. Box: “Essentially, all models are
wrong, but some are useful.”
• Some models are more useful than others
• Everyone uses a model – even if it is intuition or “common sense”
• So the question is not whether a model is “right” or whether to
use a model at all
• Question is whether one model measurably outperforms another
• A proposed model (quantitative or otherwise) should be preferred
if the error reduction compared to the current model (expert
judgment, perhaps) is enough to justify the cost of the new model
Copyright HDR 2008 dwhubbard@hubbardresearch.com
39. Total Cost Growth for Two Space
Programs (David Graham, NASA)
Development Growth Causes
25%
11%
25%
11%
11%
9%
8%
Requirements
Generation & Translation
Budget/Funding
Cost Estimation
Underestimation of Risk
Schedule Slips (Govt &
Contractor)
Price Increases
Other
Quantitative Framework
5 “The Success Triangle of Cost, Schedule, and Performance: A Blueprint for Development of Large-Scale
Systems in an Increasingly Complex Environment” - (Booz|Allen|Hamilton, 2003)