QSRA: Introduction for risk and planning professionals and decision makers webinar Wednesday 6 March 2019 presented by Tony McDonald, James Kimmance and Graham Nicol The link to the write up page and resources of this webinar: https://www.apm.org.uk/news/qsra-introduction-for-riskplanning-professional-and-decision-makers-webinar/ This webinar is sponsored by Safran Software www.safran.com

1. QSRA: Introduction for risk/planning
professionals and decision-makers
Tony McDonald
James Kimmance
Graham Nicol

2. Welcome and Introduction
• Welcome on behalf of the Risk & PMC SIGs
• Today’s presenters
– Tony McDonald, Costain
– James Kimmance, Jacobs
– Graham Nicol, Turner and Townsend
• Many thanks to Maya and Anna from APM and
members of both SIGs
• Risk SIG Conference 2019 - Using Risk to Influence
the Future (26 Mar 19 in Birmingham)

3. Why is QSRA best practice and what do decision-
makers need from it?
• QSRA answers 3 key questions concerning overrunning project schedules
– Can overruns be predicted with some degree of accuracy?
– Can causes of risk be identified upfront, helping project to stop them happening
through effective response actions?
– Can risk professionals determine causes of why projects frequently overrun their
initial schedules?
• Overwhelmingly the answer is yes!
• Addresses head-on the fact that don’t know how long activities will take
• Allows investigation of activity ‘uncertainty’ to establish its effects on a project schedule
– How likely is the project to achieve the key dates & milestones in the schedule?
– How much schedule contingency is needed to provide acceptable degree of
certainty?
– Where is the greatest risk in the schedule?
• QSRA outputs provide a strong indication to key decision-makers of the key activities
and risks that will impact the project if not managed correctly project
References: ‘Practical Schedule Risk Analysis’ Hulett, David 2009’

4. QSRA is based on a good project schedule
• Scheduling is a challenging discipline
• Vital that a project’s risk and scheduling functions work closely together to
conduct QSRA, fully involving project managers/decision-makers from the start
– Build high-level analysis network derived from the project schedule
– Utilise schedulers knowledge of project schedule to ensure that all areas
are explored for potential risk
• QSRA high-level analysis network ‘criteria’
– Left-to-right, logic driven
– ‘Finish-to-Start’ or ‘Start-to-Finish’ predecessors & ‘Finish-to-Start’ or
‘Finish-to-Finish’ successors
– Not a ‘rolled up’ project schedule
– Overall activity durations/dates match project schedule
– Minimal/no use of constraints especially Late-Date constraints
– No dangling logic ‘danglers’
– Minimal/no use of lag/leads(‘negative lag’)
References: ‘Practical Schedule Risk Analysis’ Hulett, David 2009’

5. Quantitative Schedule Risk
Analysis
Uncertainty and Risk

6. Content
• Introduction
• A Simplified QSRA Process
• Project Uncertainty and Risk
• Cautionary observations
• The benefits The biggest uncertainty in risk
analysis is whether we started
off analysing the right thing and
in the right way

7. Introduction- QSRA objectives
• Quantitative Schedule Risk Analysis primarily
used to:
– provide risk based confidence forecasts of key
milestones
– support quantification of a projects schedule (and
cost) contingency requirements
– ......also drives other ‘good’ project behaviours

8. Introduction (contd)
• A wide variety of tools are available:
• Oracle PRA / Acumen Fuse Risk/ Safran Risk
• A large volume of quality literature and guides
are available
• Hulett, Hillson, GAO (USA), APM etc
• Aim is to provide a:
– ‘Quick Start’ introduction
– Do’s, Maybe’s and Don’ts

9. A Simplified QSRA Process
• Key simplified stages – appear straightforward
Schedule
Preparation
Assign
Activity
Uncertainty
MapRisk(s)
toActivities
ApplyRisk
Correlations
Incorporate
Probabilistic
Branching
Apply
Weather
Risk
Schedule
Health
Check
Run
Analyses
Review
Analyses
andModel
Interpretive
Report
* Not covered here

10. Maybe Not That Simple
• Schedule suitable for QSRA may need a ‘logic’
rework:
– Constraints (e.g. expected finish); lags etc
– Size/level ideally 200 – 500 activities, but 2000 possible
• Many things in ‘Risk’ are intuitively understood -
but in practice need clarification. Risk and
Uncertainty are two terms that often lack
clarity, consistency in application, and are often
confused / seen as interchangeable.
Schedule
Preparation
Assign
Activity
Uncertainty
MapRisk(s)
toActivities

11. Risk & Uncertainty
• A lot has been written in the Project (Risk)
Management space differentiating between
Risk & Uncertainty
• Defining Risk
• Many ‘variations’ – but on a theme with a large consistency
• APM PMBOK.. ‘an uncertain event or condition that, if it occurs,
has a positive or negative effect on the project objective’
• Typically: event based; quantifiable with probabilities of
occurrence; and impact on the project

12. Risk & Uncertainty (contd.)
• Defining Uncertainty….
• Uncertainty definitions in Project Management are less ubiquitous
and more nebulous
• “Uncertainty is a lack of certainty”; …….”uncertainty is understood
in its most basic form as not knowing for sure, due to lack of
information or ambiguous information”;…………
• Common themes are: absence of information; unknowing;
variability; ambiguity; and lack of clarity; a degree of ‘randomness’
about an average

13. Risk vs. Uncertainty
• For practical day to day purposes:
– A risk is a discrete event with a probability of
occurrence. The risk effect is only felt if the event
occurs
– Uncertainty is an occurrence that has no
probability, and you know that you don’t know the
actual value
• Effectively a normal activity i.e. – but for which there is
an inherent ‘variability’ caused by a variety of factors
• The variability can be captured as a range based on
‘experience’

14. Creating the model and data
• Key players are the Scheduler, Risk Manager,
Subject Matter Experts, and Project Managers
– Review and amend the schedule with logic in
mind (using schedule health check tool)
– Estimate activity uncertainty
• Individual activities – SME knowledge and experience –
or maybe even data sets from similar activities
• Possibly in groups of similar activities (quick templating)
for example Civils, M&E, ICT, New Technology,
Integration and Commissioning

15. Creating the model and data (contd.)
– Determine and map schedule impacting risks
• Assumptions analysis – capture necessary risks
• Review existing risk register
• Determine probabilities, and distribution type
– Typically Triangular, or Trigen, Uniform and BetaPert
• Map to activities in the schedule model
• Doing this properly requires significant
resources
• But there are significant benefits – at all stages of the
project

16. Cautionary Observations
• The schedule and uncertainty data are the key
building blocks upon which the QSRA relies:
• Don’t skimp on creating a suitably sized schedule, with
logic suitable for an analysis
• Look for and capture assumptions
• Prepare, and then challenge activity uncertainty with
SME’s
• Risks
• Don’t just ‘drag and dump’ risks from a general risk
register into the model

17. Cautionary Observations (contd.)
• With a QSRA model - do not
• Leave the process until last / when you have a ‘finished’
schedule
– It will be rushed / tick box exercise
– You will not get many of the benefits of investing
– May suggest you have a ‘potential’ nightmare (typically just
before you have to submit for finding approval)
• Miss out the SME’s and PM’s because they are ‘busy’
• Try to ‘game’ the result
– Often reflected in some strange outputs and reports

18. Benefits
• QSRA’s can be resource consuming so
maximising benefits are important.
• Even at an early stage:
– Overruns can be associated with a lack of in depth
analysis / challenge at the planning (or even
bidding) stage. When we can more easily change
things………..
– Unrealistic or overly optimistic plans identified.
– Going through the process can identify what you
don’t know (and need to focus on) as much as
what you think you know.

19. Benefits (contd.)
• Subsequent stages
• We identify those activities and risks which are having
the biggest impact on our schedule (and costs)
• Prioritise and support the development of mitigation
activities

20. Interpretation of QSRA Outputs

21. Interpretation of QSRA Outputs
• The interpretation of QSRA Outputs and how
they are reported is probably the most critical
aspect of the QSRA process
• A lot of work will have gone into producing
the outputs
• However, the QSRA outputs can be extremely
sensitive if presented without context or
without challenge!

22. Challenges with QSRA Outputs
• Key messages can often be hidden away in
lengthy QSRA reports
• Single Pnumbers i.e P50 confidence – Date ‘X’
don’t actually inform a decision
• Decision makers struggle to grasp what QSRA
outputs are telling them and the value of the
QSRA process is often lost on them
• Often Practitioners can’t explain what’s driving
the analysis or what’s gone into the model

23. Interpretation of QSRA Outputs- Back
to Basics
• Why do QSRAs in the first place?
– Inform contingency discussions/arrangements
– Provide confidence and assurance regarding
schedule assumptions and integrity
– Identify key risks to delivery and highlight
sensitive areas within the project schedule
– Provides a range of outcomes
– Model various scenarios

24. Risk Practitioner Questions to Ask…
When questioned by decision makers/senior
management re. QSRA outputs…can I:
– Detail the quality of inputs that went into the model i.e.
management signed off schedules, uncertainty and risk
inputs
– Outline what scenarios have been modelled and any
assumptions and exclusions
– Demonstrate that the QSRA process was iterative and
involved all key stakeholders
– Agree what are the base dates, milestones etc in the
schedule that the QSRA outputs should be compared
against…doesn’t always have to just be the end date!

25. QSRA Outputs-Distribution Outputs (Pre-Mit)
Pre-mitigated position (assuming no mitigation in place or effective)
1% chance of hitting the base date (Gold arrow)
P50 output (Blue arrow)- 10/04/2024…some 534 days after base date!

26. QSRA Outputs-Distribution Outputs (Post-Mit)
Post- mitigated position (assuming identified mitigation successful!)
26% chance of hitting the base date (Gold arrow)
P50 output (Blue Arrow)- 25/07/2023…now 274 days after base date an
improvement of 260 days!

27. QSRA Outputs - Distribution Analyser
Plan Outcomes with
Uncertainty and Pre-Mitigated
RiskPlan Outcomes with
Uncertainty and Post-
Mitigated Risk
Potentially 260 days saving
if the proposed mitigations
are successful at P50!

28. QSRA Outputs - Tornado Graphs
What’s causing these
top 2 risks to drive the
QSRA model?
Why are delivery to site
activities so sensitive to
uncertainty and risk?

29. QSRA Outputs – Scatter Graphs

30. Interpretation of QSRA Outputs
• Be clear on what has gone into the model in terms of
quality inputs
• Present ranges rather than Single Pnumbers…
– P50 date is ‘X’ but considering the P75 output would give an
additional 2 weeks time risk allowance?
• Present the delta of the analysis compared to base data
• Be clear on pre and post-mitigated positions
– Why are certain risks more prominent than others?
– What drives reduction in exposure?
– Are mitigations robust to merit the reduction in exposure?

31. Q&A
Questions?

32. Thank You

33. This presentation was
delivered
at an APM webinar
To find out more about upcoming webinars
please visit our website
www.apm.org.uk/events