South Wales and West of England Branch event held on 26th May 2016. Presented by Martin Gosden, SWWE Branch Chairman

1. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Use of a Risk Maturity Model
to Benchmark Project Health
An exploration of the practical application of a risk audit
model to benchmark project (and business) health and
improve forecast cost and schedule out-turn
1
Mark Lee
Principal P3M Consultant
26 May 2016

2. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Agenda
• Introduce the principles of risk maturity assessment
– Why a mature approach is important to project and programme cost and schedule control
• Explore and explain the QinetiQ Risk Maturity Model (QRMM)
– Context and history: development of the model
– Construct and scope
– Why this particular model offers advantage to projects/programmes and businesses
– How it is applied in practice to benchmark projects/programmes and organisations
• Demonstrate the value of the QRMM in application
– Summary case examples from Defence and Oil & Gas
2

3. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Introducing QinetiQ
• Formed in 2001 from Defence Evaluation and Research Agency (DERA)
• FTSE 250 - £1.35bn market capitalisation
• 6,250 people worldwide, with over 1,000 specialists in weapons and testing
• Founding Member of The 5% Club – investing in graduates and apprentices
• 37 sites across the UK – from Cape Wrath to Shoeburyness
• 25 year Long Term Partnering Agreement (LTPA) with MOD, signed in 2006
• Empire Test Pilots’ School (ETPS) – training flight test professionals for over 70 years
• More than 1,500 patents granted
3

4. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Advisory Services – P3 Consultancy
• We are QinetiQ’s leading Cost, Risk and P3 capability
• We provide independent, impartial advice to governments and industry
• We work in Land, Air, Maritime and Joint Domains
• We deliver innovation, supporting complex and often leading edge research and
delivery programmes for international Defence and Commercial clients
• Our products and services include
– Our QinetiQ proprietary P3 tools: QRMM, CEHC and FACET
– Project Management Office services and support, risk management, cost management,
financial forecasting, resource management, stakeholder mapping and engagement, benefits
mapping, business and P3 processes and governance
4
We enable our clients to make better decisions, based on good evidence
and robust analysis, using a variety of proven tools and techniques

5. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Risk Maturity Principles
Why a mature risk management approach is
important to schedule and cost control
5

6. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Why is P3 Risk Maturity Important?
Examples of ‘Failed’ Commercial Projects
Thermae Bath Spa
Forecast: £13m and 2002 opening
Actual: £45m and 2006 opening
Scottish Parliament
Forecast: £10-40m and 2001 opening
Actual: £414m and 2004 opening
6
Source: Wikimedia.org

7. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Control of Risk Management Maturity
Schedule Impact – QinetiQ Analysis of Historical NAO Data
Current Schedule Performance vs Original Forecast of MOD Top 20 Major Projects
41%
46%
114%
47%
32%
51%
63%
54% 53%
42%
75%
8%
10%
0% 0% 0%
11%
0%
0%
20%
40%
60%
80%
100%
120%
Typhoon(Nov-87)
StingRay(May-95)
NimrodMRA4(Jul-96)
AstuteClassSub(Mar-
97)
A400M(May-00)
Type45Destroyer(Jul-
00)
SupportVehicle(Nov-01)
NGLAAW(May-02)
Terrier(Jul-02)
NavalEHF/SHFSat
Comms(Aug-03)
Soothsayer(Aug-03)
MTADS(Sep-04)
Watchkeeper(Jul-05)
Falcon(Mar-06)
Merlin(Mar-06)
FutureLynx(Jun-06)
AdvancedJetTrainer
(Aug-06)
TyphoonFuture
Capability(Jan-07)
Scheduleoverrunas%oforiginalforecast
Risk Maturity Uncontrolled Risk Maturity @ Level 3+
Forecast schedule
overrun calculated from
the summary of post-
Main Gate projects in
NAO Major Projects
Reports
Many factors affect
projects, but those with
risk maturity applied at
all CADMID stages are
more aware of issues and
have mitigations in place
to respond to those risks
Major projects from the
NAO reports with risk
maturity applied are
statistically less likely to
experience schedule
overruns
Schedule Performance vs Original Forecast of MOD Major Projects
Source data: NAO Major Projects Reports
Average schedule
over-run = 56%
Average schedule
over-run = 4%
7

8. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Control of Risk Management Maturity
Cost Impact – QinetiQ Analysis of Historical NAO Data
Forecast cost overrun
calculated from the
summary of post-Main
Gate projects in NAO
Major Projects Reports
Projects with Risk
Maturity applied
experience less budget
volatility (overspend or
underspend), compared
with projects whose level
of risk maturity is
uncontrolled
Current Schedule Performance vs Original Forecast of MOD Top 20 Major Projects
-21%
28%
48%
0%
29%
-7%
-18%
6%
-26%
42%
-7%
-1%
-5%
-1%
1%
-5%
-2%
-30%
-20%
-10%
0%
10%
20%
30%
40%
50%
Typhoon(Nov-87)
StingRay(May-95)
NimrodMRA4(Jul-96)
AstuteClassSub(Mar-
97)
A400M(May-00)
Type45Destroyer(Jul-
00)
SupportVehicle(Nov-01)
NGLAAW(May-02)
Terrier(Jul-02)
NavalEHF/SHFSat
Comms(Aug-03)
Soothsayer(Aug-03)
MTADS(Sep-04)
Watchkeeper(Jul-05)
Falcon(Mar-06)
Merlin(Mar-06)
FutureLynx(Jun-06)
AdvancedJetTrainer
(Aug-06)
TyphoonFuture
Capability(Jan-07)
Currentoverspendas%oforiginalforecast
Risk Maturity Uncontrolled Risk Maturity @ Level 3+
Budget Performance vs Original Forecast of MOD Major Projects
Source data: NAO Major Projects Reports
Average cost
overspend = 8%
Average cost
underspend = 3%
8

9. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Why is P3 Risk Maturity Important?
Audit Evidence from Defence
9
Source data: NAO Major Projects Report 2013
700
600
500
400
300
200
100
0
£m
Ships Combat Air Submarines Helicopters Air Support Information Land ISTAR Weapons
Systems Support Equipment
0 0
Defence – average cost growth across all 73 projects in the ships, combat air and submarine sectors

10. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Why is P3 Risk and Cost Maturity Important?
Avoiding the News Headlines!
10
“BBC Chief Technology Officer sacked over failed
£100m Digital Media Initiative”
“A major £12m scheme to
digitise patient records has so
far failed to deliver and is now
expected to cost the Health
Department double its original
budget, according to the Public
Accounts Committee”
“Canadian provincial health officials fire
prime contractor after the firm missed 3
years of deadlines and failed to deliver the
province’s flagship online medical registry”
“Sacked bursar loses claim for unfair dismissal
over school’s £2.5m expansion project”
“Environment Minister demoted after bungled $2.4bn home insulation fiasco”
Source: Pressgazette.co.uk

11. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Why is P3 Risk and Cost Maturity Important?
Avoiding the News Headlines!
11

12. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Why is P3 Risk and Cost Maturity Important?
Primary Causes of Project Failure
12
“Opportunities and risks were not defined” 31%

13. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Why Does This Happen?
Dealing with Uncertainty
All projects and programmes have uncertainties and changing variables, arising from …
• Budget changes
• Schedule changes
• Requirement changes
• Omissions and errors
• Failure to tackle risk at source
• Things that “just go wrong”
“…… because as we know, there are known knowns; there are things we know we know.
We also know there are known unknowns; that is to say we know there are some things
we do not know. But there are also unknown unknowns - the ones we don't know we
don't know” Donald Rumsfeld
Source: Wikimedia.org
13

14. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Why Does This Happen?
Dealing with Uncertainty – Defence Reform and QDC

15. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Why Does This Happen?
Dealing with Uncertainty – Defence Reform and QDC

16. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Confidence … Driven by Uncertainty
Confidence is a major factor in human behaviour … and decision-making
• Sub-prime mortgage crisis – Missouri house prices crashed by 90%
• Global recession - US companies began storing cash
• Scottish Referendum – investors sold Sterling
• ‘Brexit’ – value of Sterling and FTSE depressed(?), investments on hold
Wouldn’t it be useful if we could have better confidence in
our ability to manage uncertainty…?
Source: Edinburghnews.com
16
Source: Dilbert.com

17. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
The QRMM
Construct, context, scope, benefits and
application of the QinetiQ Risk Maturity Model
17

18. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QinetiQ Risk Maturity Model (QRMM)
A Brief History …
• Developed by QinetiQ (1999) to improve confidence by objectively assessing risk management
maturity
• Referenced to and compliant with
― MOD’s Acquisition System Guidance (ASG)
― Project risk management best practice – APM Project Risk Analysis and Management (PRAM) Guide
― Combined Code (‘Turnbull Guidance’) for UK Corporate Governance – Financial Reporting Council
• Proven capability and value in application over 15 years
― Circa £100bn of Defence projects/programmes (across all domains) assessed … and counting
― Also deployed in Oil & Gas (FTSE 100 multi-national), Rail and Manufacturing
• QinetiQ analysis of NAO Major Projects Reports has indicated that RMM can
― Increase confidence in project success through improved cost/schedule adherence
― Deliver forecast improvement in schedule and cost out-turn on major projects and programmes
18

19. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM as an Enabler to Better P3 Delivery
• Assesses and benchmarks the quality and consistency of risk management implementation
• Improves confidence in the ability to predict and deliver against schedule and cost
• Establishes an independent, objective and evidence-based baseline measure of risk maturity
• Identifies strengths and weaknesses in risk management process and its enablers
• Supports formulation of a prioritised ‘roadmap’ of improvement actions against the baseline
• Supports identification of common issues across projects/programmes, to help tackle risk at source
• Facilitates sharing of good practice within and across business units
• Builds confidence in the quality of underpinning data (e.g. for Business Cases)
• Scalable: applicable at all levels, at all points in the project/business lifecycle
• Can be used to support supplier assessment
19

20. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Comparison of Risk Management Maturity Models
QinetiQ Analysis (2013) - Summary
20
Defence
Heritage
Risk-Specific
Maturity
Model
Implementation
Guidance
Available
Implementable
Risk Maturity
Model Available
Question and
Anwer Set
Developed and
Available
Questions
Mapped to
Maturity
Levels
Analysis and
Reporting
Tools
Available
Improvement
Roadmap
Guidance and/or
Tool Available
Number of
Questions
Implementable
Without
Investment
Management of
Risk (MOR®)
Maturity Model
0
HM Treasury Risk
Management
Assessment
Framework
(RMAF)
38
Cabinet Office
Portfolio,
Programme and
Project
Management
Maturity Model
(P3M3)
9
QinetiQ Risk
Maturity Model
(QRMM)
50

21. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM Construct – Scope and Scoring
• A maturity framework covering 6 risk management perspectives
— Risk Identification; Risk Analysis; Risk Mitigation
— Project Management; Stakeholders; Culture
• Each perspective is scored within the algorithm, at one of 4 levels
— Level 1 = Naive (process design or application flawed and probably not adding value)
— Level 2 = Novice (some value-add, but weakness in process design or implementation)
— Level 3 = Normalised (formalised process, implemented systematically and adding value)
— Level 4 = Natural (applied at strategic level in driving objectives and optimising outcomes)
• Level 3 is an acceptable score (complex programmes/projects may aspire to Level 4)
21

22. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM Construct – Top Level Output
Risk Maturity Assessment
Stakeholders Risk
Identification
Risk Analysis Risk
Mitigation
Project
Management
Culture
Current
Maturity
Potential
Improvement
Level 4 -
Natural
Level 3 -
Normalised
Level 2 -
Novice
Level 1 -
Naive
“An organisation is only as strong as its weakest element”
22

23. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM – Example Characteristics of Maturity Levels
23
Level Descriptor Formal Definition Example Characteristics
1 Naïve
Although a risk management process
may have been initiated, its design or
application is fundamentally flawed
At this level, it is likely that the
process does not add value
• Low level of understanding of risk management principles and application
• No formalised process for managing risk, or ad-hoc and/or poorly applied
• Fundamental flaw in the design or application of the risk process
• Critical risk process components are likely to have lapsed into disuse
2 Novice
The risk management process
influences decisions taken by the
organisation in a way that is likely to
lead to improvements in performance
as measured against its objectives
However, whilst the process may add
value, weaknesses with either the
process design or its implementation
result in significant benefits being
unrealised
• At least a ‘light’ application of the risk process
• The process itself is standardised and followed with robustness
• A formal risk management process that follows best practice recently initiated
• Larger organisation where process application may be an issue
• Larger organisation where issues of process design may be difficult to correct

24. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM – Example Characteristics of Maturity Levels
24
Level Descriptor Formal Definition Example Characteristics
3 Normalised
The risk management process is
formalised and implemented
systematically
Value is added by implementing
effective management responses to
significant sources of uncertainty that
could affect the achievement of
business objectives
• Disciplined risk process application across the whole organisation
• Risk register underpins routine reviews of the implications of risk
• The effectiveness and implementation of responses is designed to manage risk
• Risks clarify all relevant and significant sources of uncertainty
• The risk register contains the right risks (and they continue to be the right risks)
• Risks are managed by the right risk owners, with engagement of stakeholders
• Appropriate and sound methods are used to select and prioritise risks for review
• Risk management is being used to support achievement of objectives
4 Natural
The risk management process leads
to the selection of risk-efficient
strategic choices when setting
business objectives and choosing
between options for solutions or
delivery
Sources of uncertainty that could
affect the achievement of objectives
are managed systematically within
the context of an organisational
culture that is conducive to optimising
business outcomes
• Risk is managed from a strategic (not just tactical) perspective
• Risk responses are likely to be executed from Sponsor level
• Risk management is implicit, with over-reliance on the Probability Impact Matrix
and an integrated risk register and Monte-Carlo simulation toolset avoided

25. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM in Application
Empirical Assessment Process
Review
Documentary
Evidence Audit
Conduct RMA
Workshop
Analyse and
Report Results
Implement
Improvement Plan
RMA Benchmark
(Level 1 to 4)
Periodic re-assessment against current benchmark
25

26. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM in Application
Risk Maturity Assessment Workshop – Use of Electronic Voting
• Well-established method of group decision support
• Used to elicit opinion
— Primarily interested in the reasons for the votes
— Votes are anonymous
• Provides a framework to consider arguments before expressing opinion
• Discussion is limited to clarification before voting
— Understand the question and supporting narrative, in relation to risk maturity
— Understand how the question and context relates to the project under assessment
• Divergence in votes may provide additional insight: an opportunity for discussion
Source: Wikipedia.org
26

27. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM in Application
Risk Maturity Assessment Workshop – Use of Electronic Voting
• Question is posed
— Consider the question and context ... and vote
• Facilitated discussion
— Voting results presented
— Salient points recorded for analysis/reporting
— Record the consensus view (score, any narrative)
• Re-vote (as necessary)
• Why use Delphi Technique?
— Decisions from a structured group facilitation are more accurate
— In this scenario, exploring voting rationale can aid interpretation and understanding
Source: Powervote.com
27

28. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM in Application
Example Analysis Outputs
28

29. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM in Application
Example Analysis Outputs
29

30. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM in Application
Example Analysis Outputs
30

31. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM in Application
Example Analysis Outputs
31

32. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM in Application
Improvement Roadmap Implementation – Example Status Check
32
Recommendation Title Theme Value Complexity Status Comments
R3.1 Risk Descriptions Process Alignment High Low Ongoing Will be satisfied through RMP
R3.3 Risk Roles and Responsibilities in RMP Process Alignment High Low Ongoing Will be satisfied through RMP
R3.4 Briefing of Risk Roles and Responsibilities Stakeholder Engagement High Low Ongoing Briefed through RMP and risk training
R4.4 Use of Risk Reduction Strategies Process Alignment High Low Ongoing Will be satisfied through RMP
R5.2 DOR Matrix Roles and Responsibilities High Low Ongoing Will be satisfied through RMP
R1.2 Escalation Criteria Process Alignment High Medium Ongoing Will be satisfied through RMP
R3.10 Secondary Risk Effects in RMP Process Alignment High Medium Ongoing Will be satisfied through RMP
R3.11 Training on Secondary Risks Stakeholder Engagement High Medium Not started To be covered in risk training course
R4.10 Risk Response Metric Audit and Assurance Medium Low Complete Already being implemented and ought to be reflected in RMP
R4.11 XXXXX Risk Board TORs Audit and Assurance Medium Low Complete TORs have been drafted and will be included in RMP
R4.2 Review of Risks for Fallback Decision Points Audit and Assurance Medium Medium Not started Suggest this is part of risk audit, planned for March 2016
R2.12 Risk Identification Metric Process Alignment Low Low Deleted Not required - risk status checked pre-Risk Board
Recommendation Title Theme Value Complexity Status Comments
R2.3 Top-Down Risk Identification in RMP Process Alignment High Low Not started Suggest this is written into RMP
R2.14 Issue Management Process Alignment High Low Not started Suggest this is written into RMP
R1.13 Portfolio Office Risk Support to DLODs Resourcing High Medium Deleted XXXXX Programme has determined this isn't needed
R2.1 Assumptions Risk Identification High Medium Not started Consider bringing MDAL and 3OA under Programme control
R2.2 Top-Down Risk Identification Risk Identification High Medium Not started Risk identification workshop scheduled for 4 November 2015
R2.13 Milestone Schedule Reviews Process Alignment High Medium Not started
R4.6 Use of Risk Tolerance Threshold Process Alignment High Medium Not started
CATEGORY 1 RECOMMENDATIONS
CATEGORY 2 RECOMMENDATIONS

33. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Case Examples
Examples of the QRMM in application to
benchmark and improve project and programme
performance in Defence and Oil & Gas
33

34. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM in Application Case Example 1
MOD Project Team – Analysis Summary and Plan
34
Perspective Number of Projects at Each Level – Feb ‘14
Level 2 Level 3 Level 4
Stakeholders √ √ √ √
Risk Identification √ √ √ √
Risk Analysis √ √ √ √
Risk Responses √ √ √ √
Project Management √ √ √ √
Culture √ √ √ √
• February 2014 forward improvement plans, focused to achieve
— Project A: from high L2 (almost L3) to weak L3 in 3 months, consolidating to a firm L3
— Project B: from high L3 to a weak L4, consolidating to a firm L4 through secondary actions
— Project C: from weak L3 (with risk of slipping back to L2) to a firm L3
— Project D: from weak L4 (risk of slipping back to L3) to a firm L4
A good example of where focused MOD effort, and periodic RMA, can enhance risk execution

35. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM in Application Case Example 2
MOD 1* Portfolio – Analysis Summary
Current Schedule Performance vs Original Forecast of MOD Top 20 Major Projects
-21%
28%
48%
0%
29%
-7%
-18%
6%
-26%
42%
-7%
-1%
-5%
-1%
1%
-5%
-2%
-30%
-20%
-10%
0%
10%
20%
30%
40%
50%
Typhoon(Nov-87)
StingRay(May-95)
NimrodMRA4(Jul-96)
AstuteClassSub(Mar-
97)
A400M(May-00)
Type45Destroyer(Jul-
00)
SupportVehicle(Nov-01)
NGLAAW(May-02)
Terrier(Jul-02)
NavalEHF/SHFSat
Comms(Aug-03)
Soothsayer(Aug-03)
MTADS(Sep-04)
Watchkeeper(Jul-05)
Falcon(Mar-06)
Merlin(Mar-06)
FutureLynx(Jun-06)
AdvancedJetTrainer
(Aug-06)
TyphoonFuture
Capability(Jan-07)
Currentoverspendas%oforiginalforecast
Risk Maturity Uncontrolled Risk Maturity @ Level 3+
Stakeholders
Risk Identification
Risk Analysis
Risk Responses
Project Management
Culture
Category1 Category2
Level 1 Level 2 Level 3 Level 4
Level of maturity after
• Level 1 across all 6 perspectives – the worst ever RMA score recorded by QinetiQ!
• Risk improvement roadmap established to target
— Level 2 in 3 months (22 actions)
— Level 3 in a further 9 months (16 actions)
Source: MoD
35

36. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM in Application Case Example 2
MOD 1* Portfolio – Outcomes
Source: MoD
• Improvements were not implemented, due to
— Lack of capacity within MOD to implement the plan
— Conflicting demands and changing priorities
— Ongoing organisational uncertainty
— Realisation that implementation of improvements at 1* level would be insufficient
• QinetiQ was then requested to
— Formulate a transformation programme covering the 2* group (4 x 1* units)
— Develop and roll-out improvements to enhance risk and cost management and control
• What happened next?
• MOD secured stakeholder buy-in to implement the ~18 month transformation programme
• In-FY underspend was secured to fund the initial phase
• Other areas developed an interest in improving their risk and cost maturity
A good example of where a localised RMA can uncover a need for wider risk/cost improvement
36

37. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM in Application Case Example 2
Example Transformation Programme – Benefits Realisation
Forecast Risk
Maturity
(left-hand scale)
Phase 1 Phase 3Phase 2
Potential for Schedule/Cost
Overrun
(right-hand scale)
Naïve (Level 1)
 Risk process flawed
 No real value-add
Novice (Level 2)
 Risk process influencing decisions
 Risk process adding value
 Improving performance against objectives
 Some process/implementation weaknesses
 Potential for significant unrealised benefits
Normalised (Level 3)
 Risk process formalised
 Process implemented systematically
 Effective risk responses executed
 Sources of uncertainty under control
 Significant value-add
Natural (Level 4)
 Risk process informing
strategic choices
 Sources of uncertainty
managed systematically
 Risk culture conducive to
maximising outcomes
Level 1
Level 4
Level 3
Level 2
RiskMaturityLevel
0-10%
100%+
60-99%
11-60%
Schedule/CostOverrun
(asa%ofbaselineschedule/budget)
37

38. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM in Application Case Example 3
MOD HQ Programme - Results
38
• A high performing programme with excellent structural and cultural characteristics
• Key shortfalls noted in risk analysis due to personnel failing to record this in the risk tool
• Risk improvement roadmap established to target
— Clear consolidation at Level 3 (49 actions), a significant number of which were RMP shortfalls
— Longer-term improvement to Level 4 (45 actions)

39. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM in Application Case Example 3
MOD HQ Programme - Outcomes
39
• What happened next?
— The programme enhanced its internal capability through the recruitment of additional P3M resource
— QinetiQ’s contract was extended to upskill the new P3M team and advise on improvement execution
— The programme targeted a further calibration of its risk management maturity in circa 6 months
A good example where even a risk-robust programme can benefit from targeted risk improvement
• QinetiQ was advised of another programme (under the same SRO) which required calibration
− QinetiQ briefed the benefits of conducting a formal risk maturity calibration to the other PMO
− The (new) PMO Head of Risk is introducing changes, having recognised some shortfalls
− QinetiQ will be contracted to undertake a risk maturity assessment in circa 3-6 months

40. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
QRMM in Application Case Example 4
Oil & Gas FTSE 100 Multinational – Analysis and Outcomes
40
• FTSE 100 Oil & Gas multi-national approached QinetiQ to pilot an RMA on a UK project
• QinetiQ amended the RMA framework Q&A set to reflect O&G-specific language
− Underlying model and algorithms were unchanged
• The pilot was conducted on the UK project
− Identified that lack of risk disclosure from the JV partner was a significant threat
− RMA was extremely well received by the client organisation
− Actions to address the shortfalls were apparently not progressed
− There were serious repercussions for both JV partners
A good example where failure to address risk maturity shortfalls can impact business health
• What happened next?
− QinetiQ undertook a further RMA on an operation in Asia, on completion of the UK pilot
− The Asia RMA identified significant pockets of good practice to share across the company
− QinetiQ was requested to develop a new corporate Cost & Schedule Risk Analysis standard
Source: Wikipedia.org

41. QINETIQ PROPRIETARY
QINETIQ PROPRIETARY
The document and information contained herein is proprietary information
of QinetiQ Limited and shall not be disclosed or reproduced without the prior
authorisation of QinetiQ Limited. © QinetiQ Limited 2016
Summary
1. Introduce the principles and importance of risk maturity assessment
− There is inherent uncertainty in all projects, programmes and businesses
− Formalised risk management help us understand and respond to uncertainty
− Control of risk maturity is a key enabler to good project and programme management
2. Explore QinetiQ Risk Maturity Model (QRMM)
− Audits and benchmarks project and programme health and focuses P3M improvement initiatives
− Enhances confidence in the likelihood of an out-turn to schedule and within budget
− Enables us to more confidently establish our risk appetite and inform strategic choice
3. Demonstrate value of QRMM in application
− Case 1 – focused improvement and periodic re-assessment enhanced project control
− Case 2 – localised RMA can trigger wider imperative to enhance risk management
− Case 3 – even projects/programmes which appear mature can benefit from targeted intervention
− Case 4 – failure to address risk maturity improvement can impact business health
Source: Pressgazette.co.uk

42. www.QinetiQ.com
FOLLOW US ON:
linkedin.com
/company/qinetiq_2
youtube.com
/user/QinetiQGroup
twitter.com
/QinetiQ
facebook.com
/qinetiqgroup