2. OUTCOMES
Understand measures and indicators that allow
the risk management process to be monitored
and controlled
Use the measurement process to improve risk
detection as well as to quantify mitigation
activities within a program
Capitalize on measurement data during program
close-out to perform causal analysis of risks and
issue causes
2
D I S T R I B U T I V E
M A N A G E M E N T
3. INTRODUCTION
All projects are subject to risk; it is inescapable. From
budget cuts and unfamiliar technology to changing
requirements and staff turnover, risks that were not
foreseen and planned for frequently cause major
project issues and even failures.
Projects that fail to manage risk, or that manage it in
an ad-hoc way, find themselves in perpetual crisis.
They never see it coming.
3
D I S T R I B U T I V E
M A N A G E M E N T
4. RISK – PROCESS VIEW (ISO)4
D I S T R I B U T I V E
M A N A G E M E N T
5. RISK – PROCESS VIEW (CMMI)
From SEI CMMI
5
D I S T R I B U T I V E
M A N A G E M E N T
6. RISK – RELATIONSHIP
From SEI “Advanced Risk Analysis for High-Performing Organizations” by Christopher Alberts and
Audrey Dorofee 2006
6
Risks are inter-related
such that sets of risks
contribute to greater
and greater impact to
the program, mission
and organization.
D I S T R I B U T I V E
M A N A G E M E N T
7. RISKS - IMPACT VIEW
From “Up the Down Escalator” by Robert Charette, ITABHI Corporation 1993
7
Risk impact to an
organization depends
on the extent to which
the organization
expects the risk to
occur.
D I S T R I B U T I V E
M A N A G E M E N T
8. BLIND SPOT #1: POWERLESS TO EVALUATE THE
RISK MANAGEMENT PROCESS
8
D I S T R I B U T I V E
M A N A G E M E N T
Problem
Number of risks by state and risk budget are not
adequate to determine the effectiveness of risk
management activities
Count-style risk measures are also too optimistic – you
need to adjust your forecast by how well you have
been performing to date
Opportunity
Develop risk information needs which address risk
management planning, execution and forecast
Provide greater visibility into the risk management
process
9. INFORMATION NEED DEFINITION
Information Need Information Product
Measurable Entity Attribute
Indicator
Derived Measure Derived Measure
Base Measure Base Measure
interpretation
model
function
method
Measurement information
model (ISO-15939 & PSM) can
be used to define information
needs that evaluate the risk
management process.
9
D I S T R I B U T I V E
M A N A G E M E N T
10. Information Product Planning Accuracy of Risk Management
Interpretation: Normal range 5 – 20%
Investigate Risk Process in range of 25 – 50%
Re-plan Risk Management when > 50%
Indicator: Escaped Risk Percent (ESCPCT)
Model: ESCPCT = 100 * (ESCCOUNT / TOTAL)
SAVEPCT = 100 * (SAVES / TOTAL)
Derived Measures: Escaped Risk Count (ESCCOUNT)
Function: ESCCOUNT = TOTAL – SAVES
Base Measure: Total Number of Defined Risks (TOTAL)
Issues That Were In the Risk Plan (SAVES)
How effective is the planning process at identifying the risks
that are being mitigated?
RISK INFORMATION NEED10
Risk escape is
similar to
approach for
defect escape
D I S T R I B U T I V E
M A N A G E M E N T
11. RISK INFORMATION NEED11
D I S T R I B U T I V E
M A N A G E M E N T
What is the current state of risks and risk cost?
Information Product Current State of Risk Mitigation
Interpretation: Normal range 0.9 – 1.05
Investigate in range: 0.75 – 1.15
Re-plan range: < 0.75 or > 1.15
Indicator: Risk Growth Performance Index (RGPI)
Risk Cost Performance Index (RCPI)
Model: RGPI = CRo / CRc
RCPI = Rac / Rec
Derived Measures: Cum Open Risk (CRo)
Cum Closed Risks (CRc)
Function: CRo = Sum of Rs(new)
CRc = Sum of Rs(closed)
Base Measure: Risks by Status (new, active, closed) (Rs)
Active Issue Mitigation Costs (Rac)
Estimated Risk Mitigation Costs (Rec)
Scale the risks
count and cost to
an index similar to
EVMS
12. 12
D I S T R I B U T I V E
M A N A G E M E N T
RISK INFORMATION NEED
What is the forecast cost of risk mitigation?
Information Product Forecasted Cost of Mitigation
Interpretation: Normal range: Estimate w/in 10% of target
Monitor range: Estimate w/in 30% of target
Re-plan range: Estimate > 30% of target
Indicator: Estimated Mitigation Costs Percent (ESTPCT)
(as a % of risk budget)
Model: ESTPCT = REMAIN / EST +
(EST * Escape Percent [from previous])
Derived Measures: Estimated Mitigation Cost (EST)
Remaining Risk Mitigation Budget (REMAIN)
Function: REMAIN = BUDGET– ACTUAL
EST = Sum (RRp * RRc)
Base Measure: Total Risk Mitigation Budget (BUDGET)
Actual Mitigation Costs To Date (ACTUAL)
Remaining Risks by Probability (RRp)
Remaining Risks by Cost Impact (RRc)
Target, not
budget – allows
management to
assume risk.
Penalty for
escaped risks to
date.
13. RISK INFORMATION NEED13
How much cost avoidance has the resources for risk
management provided?
D I S T R I B U T I V E
M A N A G E M E N T
Information Product Return On Investment
Interpretation: Acceptable range: 0.90 and above
Not acceptable range: under 0.90
Indicator: Return On Investment (ROI)
Model: ROI = Amount Saved / Amount Spent
Base Measure: Amount Saved = sum of impact costs for mitigated risks
Amount Spent = sum of resources spent on risk
14. BLIND SPOT #2: UNABLE TO IDENTIFY
RISK INDICATORS
Problem
The program risk catalog/list is not integrated into
measurement process (or other monitoring system)
Requires each risk to be examined/assessed
manually – time-consuming and error-prone
Opportunity
Review available measures and associate with risk
characteristics
Risk is less costly and more effective because the
measurement process data is “re-used”
14
D I S T R I B U T I V E
M A N A G E M E N T
15. RISK IDENTIFICATION
Evaluating a risk (for mitigation) involves analyzing current
program status for the occurrence
Analysis entails determining to what extent the conditions that
indicate the risk, called characteristics, have occurred
A single risk may have more than one set of characteristics
Risk
Evaluate
Characteristic
1
Characteristic
2
Evaluate
Characteristic
3
15
D I S T R I B U T I V E
M A N A G E M E N T
16. RISK IDENTIFICATION
Measures (e.g. base measures, derived measures and indicators) typically over-
lap with risk characteristics, that is some characteristics are already being
monitored by the measurement process
Note – Measurement process provides data for some characteristics, not the
evaluation or overall risk status/indication
base measures
derived measures
indicator Schedule
Accuracy
Milestone
completion
%
Mstones
started on
time
Mstones
finished on
time
Task
Completion
%
Tasks started
on time
Tasks
finished on
time
Risk
characteristic
16
D I S T R I B U T I V E
M A N A G E M E N T
17. SAMPLE RISK IDENTIFICATION
Risk that program oversight is not possible because the master
schedule is not accurate or reflect schedule commitments.
Map available measurement data to risk characteristics
Above, 5 of the 6 risk data elements (i.e. characteristics) are
available through the measurement process.
17
D I S T R I B U T I V E
M A N A G E M E N T
18. BLIND SPOT #3: INEFFECTIVE AT
QUANTIFYING MITIGATION
Once a risk becomes an issue, mitigation activities are initiated to
control or limit the impact. If the mitigation activities are not
monitored quantitatively, the program has no way to track the
risk trend over time and evaluate mitigation decisions.
Problem
Risk mitigation is not tracked over time to verify effectiveness
of mitigation efforts
Opportunity
Focus the measurement process on risk impact to quantify
the mitigation activities and ensure that the risk has been
closed
Determine if project plan/re-plan is warranted
18
D I S T R I B U T I V E
M A N A G E M E N T
19. QUANTIFY MITIGATION19
D I S T R I B U T I V E
M A N A G E M E N T
Measurement can
quantify risk mitigation.
Provides risk status as
part of periodic
measurement reporting.
Requirements
creep risk is
detected.
Mitigated and
closed.
20. BLIND SPOT #4: INCAPABLE OF
CAPITALIZING ON PROJECT ISSUES
Problem
During the heat of battle, it is difficult to examine root
causes of issues, risks, defects, program slips etc
After project completion/closure, the measurement process
contains data that supports different types of organizational
analysis (e.g. root cause analysis, orthogonal defect
classification)
Opportunity
Perform a program risk analysis using historic measurement
and risk data (i.e. risks mitigated during the program) to see:
Were signs of a risk present before we caught them (i.e.
expand characteristics and conditions)?
What risks occurred that were not in the risk plan?
Is the same risk being mitigated in more than one program?
20
D I S T R I B U T I V E
M A N A G E M E N T
21. CAPITALIZE ON PROJECT ISSUES
• Define critical success factors to address risk causes
• Identify candidate process changes
Ensure Process
Execution
• Modify mapping of measures to risk
• Additional analysis techniques
Monitor Processes
• Review and adjust risk catalog
• Modify characteristics and conditions
Detect Impacts
21
Historic
Risk Data
Historic
Measurement
Data Program
Risk
Analysis
Similar to defect root cause
analysis, program risk analysis
investigates what can be
changed to improve execution
and improve risk management.
D I S T R I B U T I V E
M A N A G E M E N T
22. FUTURE APPLICATIONS
The techniques presented here are targeted at the
first three levels of the risk helix: crisis
management, fix-on-failure and mitigation of
symptoms
Future work will focus measurement on the
elimination and root causes levels
Produce visualizations of risk status based on the
SEI risk relationship view
22
D I S T R I B U T I V E
M A N A G E M E N T
23. SUMMARY
The aim of risk management is not to eliminate risk,
rather it is to manage project risks to maximize
opportunities and minimize the adverse effects
of unfortunate occurrences so that performance
improves and goals and objectives are more
likely to be met.
Focusing the capabilities of the measurement
process on the blind spots of risk management
enables projects to pro-actively plan for and deal
with risk factors rather than wait for problems to
occur and then try to react.
23
D I S T R I B U T I V E
M A N A G E M E N T
