Guidance on which probability
distributions should be selected across
different levels of maturity
Sponsors:
Dr. John Ahme...
Introduction
Introduction Methodology
Literature
Review
Survey Results Data Analysis Guideline Validation
Conclusion and
F...
Methodology
Introduction Methodology
Literature
Review
Survey Results Data Analysis Guideline Validation
Conclusion and
Fu...
Background
• The concept of maturity
• Cost estimate classification systems
• Probability distributions
5
Aim and Objectiv...
Concept of maturity
6
Data maturity:
Your understanding and confidence regarding a cost element
Project maturity:
Extent a...
AACEI Classification
system
AACEI : Association for the Advancement of Cost Engineering International
Introduction Methodo...
Probability distributions
• Basics
• Properties
• Continuous vs. Discrete
• Bounded vs. Unbounded
• Correlation
Introducti...
Probability distributions
Continuous vs Discrete
9
Probability
Cost £
Continuous DiscreteVS
Introduction Methodology
Liter...
Probability distributions
Bounded vs. Unbounded
10
Probability
Cost £
Probability
Cost
£
Unbounded BoundedVS
Introduction ...
Commonly used
continuous distributions
• Uniform
• Triangular
• Beta
• Beta PERT
Probability
Cost £
MaxMin
Probability
Cos...
Correlation
Basics:
Relationship between two cost elements
Effects:
Considerable impact on the cost estimate
Introduction ...
Outcomes of literature
review
• Identified the project phases depending on maturity
• Developed a comprehensive list of di...
Capturing current
practice
Survey Aim
Understand the current practice in industry towards cost estimation, methodology
and...
Survey results I
6%
39%
2%
47%
6%
No consideration of project
maturity
Using subjective judgement rather
than formal syste...
0
5
10
15
20
25
30
In which Life cycle stage do you assess
project maturity?
Survey results II
Early stage
Project maturit...
Survey results III
'''This table represents the project maturity phases.'''
Least mature More mature
Name of the
project m...
Survey results IV
0
2
4
6
8
10
12
1 2 3 4 5
Triangular
Uniform
Normal
Beta
Lognormal
Exponential
Which probability distrib...
• No standard cost estimate classification systems are
available currently in industry.
• Lack of awareness regarding sele...
Data Analysis
Aim
Support the creation of probability distributions selection guideline.
• Cases
• Cost models from sponso...
• Define forecast
• Define assumptions
• Set simulation parameters
• Run Monte Carlo simulation
• Analyse the results
Data...
Data Analysis:
Findings I
Symmetric datasetAsymmetric dataset
• Different probability distribution could lead to different...
Data Analysis:
Findings II
Statistics Minimum Maximum 10-90% Interval
Changes - 9.23% + 10.89% + 326%
Estimation range cha...
Data Analysis:
Findings III
Comparison of outcomesTornado chart to determine the key drivers
Unsuitable distribution for t...
• Correlation should always be considered from the
beginning
• Prioritise key cost drivers
• Dataset features should be co...
Guideline Overview
Aim
Guide the cost engineer to select the most suitable probability distribution for cost
elements.
• G...
Guideline
Introduction Methodology
Literature
Review
Survey Results Data Analysis Guideline Validation
Conclusion and
Futu...
Yes
Guideline Flow chart
Adapted from: Strategic Risk Taking: A Framework For Risk Management by Aswath Damodaran (2007)
S...
Data Maturity
Triangular
Beta-PERT
Beta
Data
Maturity
Introduction Methodology
Literature
Review
Survey Results Data Analy...
Guideline Tool
• Readme
• Start
• Continuous
• Discrete
• Guidance
• Explanations
Introduction Methodology
Literature
Revi...
Guideline Demo
Is the estimate symmetric?
Yes
How confident are you
in your boundaries?
Confident
Trigen,
Normal
Do you ha...
Validation
Survey
“Think it is quite a good document!”
Section Leader, BAE Systems Submarines
Data Analysis
“No areas for ...
Conclusion
• Literature review
• No consideration of project maturity in probability
distribution selection
• Survey
• Cos...
Acknowledgements
• Andy Langridge, Price Systems
• Richard Parker, BAE Systems
• Tony Higham, BAE Systems
Thank you for your
attention!
Guidance on which probability distributions for different levels of maturity - Dr John Ahmet Erkoyonuncu
Upcoming SlideShare
Loading in...5
×

Guidance on which probability distributions for different levels of maturity - Dr John Ahmet Erkoyonuncu

242

Published on

This is one of many excellent presentations given over the last three years of the eVa in the UK series. They can also be found in the archive at: http://evaintheuk.org/archive along with back-copy video footage in http://evaintheuk/pmchannel
EVA19, the long established Earned Value conference, has this year described its theme as looking at a project management ‘ABC’ – Agile, Benefits and Complex.
The four day event, which returns to the Armourers Hall, runs from the 19th to 22nd of May with the flagship conference being held on 20th and 21st May and workshops before and after.
The conference will look at how this ‘ABC’ can be made to work within a portfolio and how agile fits into major and minor projects. It will investigate how to manage the relationship between portfolio benefits and project budgets, and whether complex projects even exist.
Conference organiser and APM chairman, Steve Wake says:
“Currently there is little evidence that this ‘ABC’ is being effectively deployed and managed. This conference aims to address that concern through EVA’s trademark blend of learning and professional development. Case studies and unusual presentations, delivered by top-notch speakers and experienced practitioners, will again engage and entertain the audience.
We’ve used string quartets to illustrate points in the past and this year we will be using a Blues band for the first time.”
Speakers across the two days include many familiar faces from the APM events programme including; Adrian Pyne of the APM ProgM SIG ‘Changing the project wasteland with a portfolio culture that works,’ APM Honorary Fellow Tim Banfield Director at the Major Projects Authority and Stephen Jones, Sellafield and Planning Monitoring and Control Specific Interest Group (PMC SIG) and Carolyn Limbert of the APM PMC SIG to talk about agile, benefits and complex.
Peter Taylor, the Lazy Project Manager will be presenting on “The project manager who smiled” and the ever popular Stephen Carver will present the leadership lessons that can be learnt from Alfred the Great.
In addition, there will be speakers from AIRBUS, TfL, Bloodhound, Heathrow T2 and London Tideway Tunnels.
The conference will be supplemented by a number of workshops being held at the Chartered Institute of Arbitrators, Bloomsbury Square on Monday 19th and Thursday 22nd May 2014.
'eVa in the UK' http://evaintheuk.org is building a reputation, brand and a learning legacy for the Project Management Profession. The event series is now in its nineteenth year. It is almost as if it all kicked-off when Steve Wake was in short trousers and knights roamed the land on their chargers!
#eva19 is an excellent example of Listening, Learning and Leading #apmLLL in action, and great opportunity for professional development.

I would encourage anyone who is interested in 'Building a better Project Manager,' to take a look at the web site, and book your place and get involved.

Published in: Business, Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
242
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Cost estimation classificationAACE classification systemProbability distributions – apply to the cost drivers and next run a montecarlo analysis, stochastic methods- Example of distributions
  • Assessement processes for a data maturity
  • As explained below, the AACEI provided a generally acceptable classification system within any industries. This five phase classification model is the basis of numerous classification models and a simplified version became the ANSI Standard Z94. This recommended guideline presents general characteristic and can be the starting point for the development of a firm-specific cost estimate guideline. It can also be associated with several addenda which provide more specific details for particular industries such as the three addenda presented in Table 3.As explained above, you can see the evolution of the expected accuracy range and of the methods along the project progress.
  • Triangular = 3 point estimatesBeta = 2 parameters Alpha, Beta Beta PERT = Used where a most likely value is clearly discerned, and the shape of the distribution is highly skewed/symmetricUniform: two pointsNormal =Lognormal =Weibull =Uniform =Exp =Maturity increase, the shape is more sophisticated
  • Imagine you have positive correlation of two cost elements, i.e., they increase or decrease together. If you fail to incorporate this knowledge in your total costs you will have a higher risk of budget over- and under run
  • Challenge: How can we select the probability distribution (fit to the data)?
  • AimsSupport the creation of probability selection guidelineCases Cost models from sponsorsBased on three points estimateDifferent maturity levelsTools and MethodologyCrystal Ball & ArriscaMonte Carlo simulation
  • Table outcomes (overview)
  • Why Ranked?
  • CHANGE THE PATH!
  • Transcript of "Guidance on which probability distributions for different levels of maturity - Dr John Ahmet Erkoyonuncu"

    1. 1. Guidance on which probability distributions should be selected across different levels of maturity Sponsors: Dr. John Ahmet ERKOYUNCU Project Controls 11th June 2013
    2. 2. Agenda • Introduction • Methodology • Literature Review • Survey Results • Data Analysis and Guideline • Validation • Conclusion and Future work Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    3. 3. Introduction Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work Probability Cost £500 700 900 1000 25% Stochastic Methods Three point estimating Run a Monte Carlo Simulation Apply a distribution to cost drivers
    4. 4. Methodology Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work • Searched key words • Identified main journals Literature Review • Create questionnaire • Conduct Survey Survey • Analyse Case Studies • Brainstorm Guideline Data Analysis • Analyse Case Studies • Create guideline tool Guideline Development Validation Deliverable 1 Deliverable 2 Deliverable 3
    5. 5. Background • The concept of maturity • Cost estimate classification systems • Probability distributions 5 Aim and Objectives Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    6. 6. Concept of maturity 6 Data maturity: Your understanding and confidence regarding a cost element Project maturity: Extent and accuracy and comprehensiveness of available information and data Aim and Objectives Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    7. 7. AACEI Classification system AACEI : Association for the Advancement of Cost Engineering International Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    8. 8. Probability distributions • Basics • Properties • Continuous vs. Discrete • Bounded vs. Unbounded • Correlation Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    9. 9. Probability distributions Continuous vs Discrete 9 Probability Cost £ Continuous DiscreteVS Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    10. 10. Probability distributions Bounded vs. Unbounded 10 Probability Cost £ Probability Cost £ Unbounded BoundedVS Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    11. 11. Commonly used continuous distributions • Uniform • Triangular • Beta • Beta PERT Probability Cost £ MaxMin Probability Cost £ Min Most Likely Max Probability Cost £ Most Likely + Standard deviation Probability Cost £Min Max + 2 shape parameters • Normal • Lognormal • Weibull • Exponential Uniform Triangular NormalWeibull Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    12. 12. Correlation Basics: Relationship between two cost elements Effects: Considerable impact on the cost estimate Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    13. 13. Outcomes of literature review • Identified the project phases depending on maturity • Developed a comprehensive list of distribution and characteristic Challenge: How to select a suitable probability distribution for cost elements across maturity phases? Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    14. 14. Capturing current practice Survey Aim Understand the current practice in industry towards cost estimation, methodology and project maturity classification systems. • Online survey • ACostE members • Link project maturity with probability distribution selection • 76% large enterprise • 80% of respondents, have more than 5 years experience in cost estimation Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    15. 15. Survey results I 6% 39% 2% 47% 6% No consideration of project maturity Using subjective judgement rather than formal system Using general cost estimate classification system Using customised cost estimate classification system Others Do you take into account the maturity of a project to develop an estimate? There is no standard for cost estimate classification system. Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    16. 16. 0 5 10 15 20 25 30 In which Life cycle stage do you assess project maturity? Survey results II Early stage Project maturity assessment seems to be critical during early project life cycle phase. Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    17. 17. Survey results III '''This table represents the project maturity phases.''' Least mature More mature Name of the project maturity Initiation/Conceptual / Scoping Feasibility Preliminary design Detailed design Definitive/Execution Percentage of project completion 0-2% 1-15% 10-40% 30-70% 50-100% Purpose of estimate at project maturity level Concept Screening Feasibility Budget, Authorization or Control Control Check Estimate or Control Expected accuracy range - 20 to 50% / +30 to 100% - 15 to 30% / +20 to 50% - 10 to 20% / +10 to 30% - 5 to 15% / +5 to 20% - 3 to 10% / +3 to 15% Project maturity classification system Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work Table is recommended for industries to develop their own project maturity classification system.
    18. 18. Survey results IV 0 2 4 6 8 10 12 1 2 3 4 5 Triangular Uniform Normal Beta Lognormal Exponential Which probability distribution do you use at different project maturity phases? • Top three probability distributions used are Triangular, Uniform and Normal. • The distribution selection does not seem to vary across phases Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    19. 19. • No standard cost estimate classification systems are available currently in industry. • Lack of awareness regarding selecting distribution • Guidelines are needed for industry. Survey outcomes Challenge: How to select a suitable probability distribution for cost elements across maturity phases? Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    20. 20. Data Analysis Aim Support the creation of probability distributions selection guideline. • Cases • Cost models from sponsors • Based on three points estimate • Different maturity levels • Tools • Crystal Ball & Arrisca • Monte Carlo simulation Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    21. 21. • Define forecast • Define assumptions • Set simulation parameters • Run Monte Carlo simulation • Analyse the results Data Analysis: Workflow Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    22. 22. Data Analysis: Findings I Symmetric datasetAsymmetric dataset • Different probability distribution could lead to different cost estimation outcomes • Symmetric dataset could have similar results with different probability distributions • Some distributions have specific properties Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    23. 23. Data Analysis: Findings II Statistics Minimum Maximum 10-90% Interval Changes - 9.23% + 10.89% + 326% Estimation range changes when correlation is considered Correlation could significantly change the cost estimation range Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    24. 24. Data Analysis: Findings III Comparison of outcomesTornado chart to determine the key drivers Unsuitable distribution for the key cost drivers could lead to much worse outcomes Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    25. 25. • Correlation should always be considered from the beginning • Prioritise key cost drivers • Dataset features should be considered • Distribution parameters should be set with caution • The selected distribution should be plotted and validated Data Analysis: Conclusion Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    26. 26. Guideline Overview Aim Guide the cost engineer to select the most suitable probability distribution for cost elements. • Guideline • Guideline tool • Guideline tool Demo Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    27. 27. Guideline Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    28. 28. Yes Guideline Flow chart Adapted from: Strategic Risk Taking: A Framework For Risk Management by Aswath Damodaran (2007) Sensitivity Analysis: Identify key drivers Apply correlation Is the data continuous? Yes Is the estimate symmetric? Yes How confident are you in your boundaries? Neg Binomial Strong positive Negative Skewness No Yes No Can you estimate outcomes and probability? Estimate probability distribution. Is the estimate symmetric? Yes No Do you have most likely value? What is the skewness of your distribution? Yes No Binomial Discrete Uniform Geometric Hypergeometric Positive Skewness Confident Very confident Less confident Triangular, BetaPERT, Beta Trigen, Logistic No Uniform Triangular, Trigen, Bet aPERT, Log normal, Wei bull, Gamm a Positive Skew Triangular, Tr igen, BetaP ERT, MAX Extreme, Exp onential No What is the skewness of your distribution? Strong positive Skew Negative Skew Triangular, Tr igen, BetaP ERT, MIN Extreme Trigen, Normal Do you have a most likely value? Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    29. 29. Data Maturity Triangular Beta-PERT Beta Data Maturity Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    30. 30. Guideline Tool • Readme • Start • Continuous • Discrete • Guidance • Explanations Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    31. 31. Guideline Demo Is the estimate symmetric? Yes How confident are you in your boundaries? Confident Trigen, Normal Do you have most likely value? Yes Is the data continuous? Yes Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    32. 32. Validation Survey “Think it is quite a good document!” Section Leader, BAE Systems Submarines Data Analysis “No areas for improvement but areas which can be expanded in future research.” Principal Reliability Specialist, BAE Systems ATC Guideline “Yes I will be looking at that. I will be promoting this with other business units who haven’t had direct visibility of this.” Principal Reliability Specialist, BAE Systems ATC Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    33. 33. Conclusion • Literature review • No consideration of project maturity in probability distribution selection • Survey • Cost estimation classification system • Data analysis • Data features are the main influence on distribution selection • Correlation assessment • Sensitivity analysis to prioritise cost drivers Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
    34. 34. Acknowledgements • Andy Langridge, Price Systems • Richard Parker, BAE Systems • Tony Higham, BAE Systems
    35. 35. Thank you for your attention!

    ×