Guidance on which probability distributions for different levels of maturity - Dr John Ahmet Erkoyonuncu
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Guidance on which probability distributions for different levels of maturity - Dr John Ahmet Erkoyonuncu

on

  • 157 views

This is one of many excellent presentations given over the last three years of the eVa in the UK series. They can also be found in the archive at: http://evaintheuk.org/archive along with back-copy ...

This is one of many excellent presentations given over the last three years of the eVa in the UK series. They can also be found in the archive at: http://evaintheuk.org/archive along with back-copy video footage in http://evaintheuk/pmchannel
EVA19, the long established Earned Value conference, has this year described its theme as looking at a project management ‘ABC’ – Agile, Benefits and Complex.
The four day event, which returns to the Armourers Hall, runs from the 19th to 22nd of May with the flagship conference being held on 20th and 21st May and workshops before and after.
The conference will look at how this ‘ABC’ can be made to work within a portfolio and how agile fits into major and minor projects. It will investigate how to manage the relationship between portfolio benefits and project budgets, and whether complex projects even exist.
Conference organiser and APM chairman, Steve Wake says:
“Currently there is little evidence that this ‘ABC’ is being effectively deployed and managed. This conference aims to address that concern through EVA’s trademark blend of learning and professional development. Case studies and unusual presentations, delivered by top-notch speakers and experienced practitioners, will again engage and entertain the audience.
We’ve used string quartets to illustrate points in the past and this year we will be using a Blues band for the first time.”
Speakers across the two days include many familiar faces from the APM events programme including; Adrian Pyne of the APM ProgM SIG ‘Changing the project wasteland with a portfolio culture that works,’ APM Honorary Fellow Tim Banfield Director at the Major Projects Authority and Stephen Jones, Sellafield and Planning Monitoring and Control Specific Interest Group (PMC SIG) and Carolyn Limbert of the APM PMC SIG to talk about agile, benefits and complex.
Peter Taylor, the Lazy Project Manager will be presenting on “The project manager who smiled” and the ever popular Stephen Carver will present the leadership lessons that can be learnt from Alfred the Great.
In addition, there will be speakers from AIRBUS, TfL, Bloodhound, Heathrow T2 and London Tideway Tunnels.
The conference will be supplemented by a number of workshops being held at the Chartered Institute of Arbitrators, Bloomsbury Square on Monday 19th and Thursday 22nd May 2014.
'eVa in the UK' http://evaintheuk.org is building a reputation, brand and a learning legacy for the Project Management Profession. The event series is now in its nineteenth year. It is almost as if it all kicked-off when Steve Wake was in short trousers and knights roamed the land on their chargers!
#eva19 is an excellent example of Listening, Learning and Leading #apmLLL in action, and great opportunity for professional development.

I would encourage anyone who is interested in 'Building a better Project Manager,' to take a look at the web site, and book your place and get involved.

Statistics

Views

Total Views
157
Views on SlideShare
157
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Cost estimation classificationAACE classification systemProbability distributions – apply to the cost drivers and next run a montecarlo analysis, stochastic methods- Example of distributions
  • Assessement processes for a data maturity
  • As explained below, the AACEI provided a generally acceptable classification system within any industries. This five phase classification model is the basis of numerous classification models and a simplified version became the ANSI Standard Z94. This recommended guideline presents general characteristic and can be the starting point for the development of a firm-specific cost estimate guideline. It can also be associated with several addenda which provide more specific details for particular industries such as the three addenda presented in Table 3.As explained above, you can see the evolution of the expected accuracy range and of the methods along the project progress.
  • Triangular = 3 point estimatesBeta = 2 parameters Alpha, Beta Beta PERT = Used where a most likely value is clearly discerned, and the shape of the distribution is highly skewed/symmetricUniform: two pointsNormal =Lognormal =Weibull =Uniform =Exp =Maturity increase, the shape is more sophisticated
  • Imagine you have positive correlation of two cost elements, i.e., they increase or decrease together. If you fail to incorporate this knowledge in your total costs you will have a higher risk of budget over- and under run
  • Challenge: How can we select the probability distribution (fit to the data)?
  • AimsSupport the creation of probability selection guidelineCases Cost models from sponsorsBased on three points estimateDifferent maturity levelsTools and MethodologyCrystal Ball & ArriscaMonte Carlo simulation
  • Table outcomes (overview)
  • Why Ranked?
  • CHANGE THE PATH!

Guidance on which probability distributions for different levels of maturity - Dr John Ahmet Erkoyonuncu Presentation Transcript

  • 1. Guidance on which probability distributions should be selected across different levels of maturity Sponsors: Dr. John Ahmet ERKOYUNCU Project Controls 11th June 2013
  • 2. Agenda • Introduction • Methodology • Literature Review • Survey Results • Data Analysis and Guideline • Validation • Conclusion and Future work Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 3. Introduction Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work Probability Cost £500 700 900 1000 25% Stochastic Methods Three point estimating Run a Monte Carlo Simulation Apply a distribution to cost drivers
  • 4. Methodology Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work • Searched key words • Identified main journals Literature Review • Create questionnaire • Conduct Survey Survey • Analyse Case Studies • Brainstorm Guideline Data Analysis • Analyse Case Studies • Create guideline tool Guideline Development Validation Deliverable 1 Deliverable 2 Deliverable 3
  • 5. Background • The concept of maturity • Cost estimate classification systems • Probability distributions 5 Aim and Objectives Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 6. Concept of maturity 6 Data maturity: Your understanding and confidence regarding a cost element Project maturity: Extent and accuracy and comprehensiveness of available information and data Aim and Objectives Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 7. AACEI Classification system AACEI : Association for the Advancement of Cost Engineering International Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 8. Probability distributions • Basics • Properties • Continuous vs. Discrete • Bounded vs. Unbounded • Correlation Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 9. Probability distributions Continuous vs Discrete 9 Probability Cost £ Continuous DiscreteVS Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 10. Probability distributions Bounded vs. Unbounded 10 Probability Cost £ Probability Cost £ Unbounded BoundedVS Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 11. Commonly used continuous distributions • Uniform • Triangular • Beta • Beta PERT Probability Cost £ MaxMin Probability Cost £ Min Most Likely Max Probability Cost £ Most Likely + Standard deviation Probability Cost £Min Max + 2 shape parameters • Normal • Lognormal • Weibull • Exponential Uniform Triangular NormalWeibull Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 12. Correlation Basics: Relationship between two cost elements Effects: Considerable impact on the cost estimate Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 13. Outcomes of literature review • Identified the project phases depending on maturity • Developed a comprehensive list of distribution and characteristic Challenge: How to select a suitable probability distribution for cost elements across maturity phases? Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 14. Capturing current practice Survey Aim Understand the current practice in industry towards cost estimation, methodology and project maturity classification systems. • Online survey • ACostE members • Link project maturity with probability distribution selection • 76% large enterprise • 80% of respondents, have more than 5 years experience in cost estimation Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 15. Survey results I 6% 39% 2% 47% 6% No consideration of project maturity Using subjective judgement rather than formal system Using general cost estimate classification system Using customised cost estimate classification system Others Do you take into account the maturity of a project to develop an estimate? There is no standard for cost estimate classification system. Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 16. 0 5 10 15 20 25 30 In which Life cycle stage do you assess project maturity? Survey results II Early stage Project maturity assessment seems to be critical during early project life cycle phase. Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 17. Survey results III '''This table represents the project maturity phases.''' Least mature More mature Name of the project maturity Initiation/Conceptual / Scoping Feasibility Preliminary design Detailed design Definitive/Execution Percentage of project completion 0-2% 1-15% 10-40% 30-70% 50-100% Purpose of estimate at project maturity level Concept Screening Feasibility Budget, Authorization or Control Control Check Estimate or Control Expected accuracy range - 20 to 50% / +30 to 100% - 15 to 30% / +20 to 50% - 10 to 20% / +10 to 30% - 5 to 15% / +5 to 20% - 3 to 10% / +3 to 15% Project maturity classification system Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work Table is recommended for industries to develop their own project maturity classification system.
  • 18. Survey results IV 0 2 4 6 8 10 12 1 2 3 4 5 Triangular Uniform Normal Beta Lognormal Exponential Which probability distribution do you use at different project maturity phases? • Top three probability distributions used are Triangular, Uniform and Normal. • The distribution selection does not seem to vary across phases Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 19. • No standard cost estimate classification systems are available currently in industry. • Lack of awareness regarding selecting distribution • Guidelines are needed for industry. Survey outcomes Challenge: How to select a suitable probability distribution for cost elements across maturity phases? Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 20. Data Analysis Aim Support the creation of probability distributions selection guideline. • Cases • Cost models from sponsors • Based on three points estimate • Different maturity levels • Tools • Crystal Ball & Arrisca • Monte Carlo simulation Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 21. • Define forecast • Define assumptions • Set simulation parameters • Run Monte Carlo simulation • Analyse the results Data Analysis: Workflow Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 22. Data Analysis: Findings I Symmetric datasetAsymmetric dataset • Different probability distribution could lead to different cost estimation outcomes • Symmetric dataset could have similar results with different probability distributions • Some distributions have specific properties Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 23. Data Analysis: Findings II Statistics Minimum Maximum 10-90% Interval Changes - 9.23% + 10.89% + 326% Estimation range changes when correlation is considered Correlation could significantly change the cost estimation range Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 24. Data Analysis: Findings III Comparison of outcomesTornado chart to determine the key drivers Unsuitable distribution for the key cost drivers could lead to much worse outcomes Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 25. • Correlation should always be considered from the beginning • Prioritise key cost drivers • Dataset features should be considered • Distribution parameters should be set with caution • The selected distribution should be plotted and validated Data Analysis: Conclusion Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 26. Guideline Overview Aim Guide the cost engineer to select the most suitable probability distribution for cost elements. • Guideline • Guideline tool • Guideline tool Demo Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 27. Guideline Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 28. Yes Guideline Flow chart Adapted from: Strategic Risk Taking: A Framework For Risk Management by Aswath Damodaran (2007) Sensitivity Analysis: Identify key drivers Apply correlation Is the data continuous? Yes Is the estimate symmetric? Yes How confident are you in your boundaries? Neg Binomial Strong positive Negative Skewness No Yes No Can you estimate outcomes and probability? Estimate probability distribution. Is the estimate symmetric? Yes No Do you have most likely value? What is the skewness of your distribution? Yes No Binomial Discrete Uniform Geometric Hypergeometric Positive Skewness Confident Very confident Less confident Triangular, BetaPERT, Beta Trigen, Logistic No Uniform Triangular, Trigen, Bet aPERT, Log normal, Wei bull, Gamm a Positive Skew Triangular, Tr igen, BetaP ERT, MAX Extreme, Exp onential No What is the skewness of your distribution? Strong positive Skew Negative Skew Triangular, Tr igen, BetaP ERT, MIN Extreme Trigen, Normal Do you have a most likely value? Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 29. Data Maturity Triangular Beta-PERT Beta Data Maturity Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 30. Guideline Tool • Readme • Start • Continuous • Discrete • Guidance • Explanations Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 31. Guideline Demo Is the estimate symmetric? Yes How confident are you in your boundaries? Confident Trigen, Normal Do you have most likely value? Yes Is the data continuous? Yes Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 32. Validation Survey “Think it is quite a good document!” Section Leader, BAE Systems Submarines Data Analysis “No areas for improvement but areas which can be expanded in future research.” Principal Reliability Specialist, BAE Systems ATC Guideline “Yes I will be looking at that. I will be promoting this with other business units who haven’t had direct visibility of this.” Principal Reliability Specialist, BAE Systems ATC Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 33. Conclusion • Literature review • No consideration of project maturity in probability distribution selection • Survey • Cost estimation classification system • Data analysis • Data features are the main influence on distribution selection • Correlation assessment • Sensitivity analysis to prioritise cost drivers Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
  • 34. Acknowledgements • Andy Langridge, Price Systems • Richard Parker, BAE Systems • Tony Higham, BAE Systems
  • 35. Thank you for your attention!