Informing Program Performance with Programmatic and Technical Risk

of 24
Informing Program Performance
with
Programmatic and Technical Risk
Version 0.1 (Draft Release)
8 June 2015
Prepared by
Glen B. Alleman
David Walden
for
Office of the Undersecretary of Defense (OUSD)
Acquisition, Technology, and Logistics (AT&L)
Performance Assessments and Root Cause Analyses (PARCA)
Earned Value Management (EVM)
Washington, D.C.

of 24
1 Purpose of this Guidebook Section ..................................................................................................................3
2 Synergy Between Risk Management and Earned Value Management ............................................................4
2.1 What is Risk and How Does It Impact Earned Value Measures?......................................................................4
2.1.1 Informing Program Performance With Risk Informed BCWP ..........................................................................5
2.1.2 DI-MGMT-81861 Schedule Risk Assessment in §3.7.1.3.6...............................................................................7
2.1.3 Program Risk Analysis Background ..................................................................................................................7
3 Building the Risk Adjusted Performance Measurement Baseline....................................................................8
3.1 Six Steps to Arrive at a Risk Informed Performance Measurement Baseline...................................................8
3.1.1 Risk Management Starts with the Work Breakdown Structure.....................................................................10
3.1.2 Finding Risks in the Integrated Master Plan (IMP).........................................................................................10
3.1.3 Creating a Risk Informed PMB from WBS to IMP ..........................................................................................11
3.1.4 Capturing Reducible Risk Starts with the Work Breakdown Structure (WBS) ...............................................11
3.2 Steps to Building a Risk Informed Integrated Master Schedule (IMS)............................................................12
3.2.1 Reducible Risk Mitigation Processes Are Placed In The IMS..........................................................................12
3.2.2 Irreducible Risk Management Placed In Schedule Margin in the IMS ...........................................................13
3.2.3 Development of Cost Margin or Contingency Reserve..................................................................................16
3.2.4 Development of Management Reserve.........................................................................................................16
3.3 Summary of Establishing a Risk Adjusted PMB ..............................................................................................16
4 Informing Current Period Performance .........................................................................................................18
4.1 Connecting the Dots During Execution of the IMS.........................................................................................18
4.1.1 Risk Retirement as a Measure of Program Performance...............................................................................19
4.1.2 Schedule Margin Burn-Down as a Measure of Program Performance..........................................................20
5 Informing Forecast Performance ...................................................................................................................22
6 References......................................................................................................................................................23

of 24
1 Purpose of this Guidebook Section
This Guidebook section provides hands on guidance to Government Program Managers and Government Program
Performance Analysts for identifying, integrating, managing, assessing, and suggesting corrective action for
program performance technical and programmatic risk impacts. These Technical, Programmatic, and Business
Events influence the program performance as shown in Figure 1. Using the Department of Defense Risk
Management Guide for Defense Acquisition Programs, [1] this section addresses how to inform Program
Performance (BCWP) with risk – what can go wrong?
Figure 1 – Reducible and Irreducible uncertainties create risks to program performance must be reflected in the Earned Value Management
data to assure EV Performance matches Technical Performance
This Guidebook describes how to use the reducible and irreducible programmatic and technical uncertainties and
the resulting risk that inform program performance. This includes how to:
1. Define the reducible and irreducible uncertainties that create technical and programmatic risk. These
include event based technical and programmatic risks and the naturally occurring variances for work efforts
and technical performance.
2. Place these risks in a Risk Register. For the Reducible Risks, define the Probability of Occurrence, activities to
Mitigate the risk, and the Residual risk after mitigation. For Irreducible Risk, define the Probability
Distribution Functions (PDF) for the naturally occurring variance.
3. Connect these risks to the products and processes in Work Breakdown Structure (WBS), Integrated Master
Plan (IMP), and Integrated Master Schedule (IMS). For reducible risks with mitigations, define work in the
IMS to reduce them or assign budget to Management Reserve to cover the probability the risk will occur.
For irreducible risks, with their probability distribution model and resulting variances, assign Schedule
Margin to protect schedule delays.
4. Identify when, where, and how to inform Earned Value (BCWP) using the reducible and irreducible risks risk
at the work performance level using program performance per Guideline XX
5. Status the reduction plans for reducible risks using the risk buy down plan and use this to inform BCWP with
variance between planned and actual risk reduction.
6. Status the performance of irreducible risk (work duration variance) with margin burn down and its impact
on future irreducible risk and any adjustments to the PDF from past performance.

of 24
2 Synergy Between Risk Management and Earned Value Management
When a program is required to comply with ANSI/EIA-748, Earned Value Management, risk
management should be integrated with EVM to expose underlying drivers of performance risk. [1]
This advice is the basis of the risk section of the Guidebook, since both Earned Value Management and Risk
Management address the same problem of program performance assessment. Both provide management
information to the decision makers. Risk Management and Earned Value Management have strengths and
weaknesses in forecasting program performance outside of the past performance Cost Performance Index and
Schedule Performance Index measures. Risk Management and Earned Value Management have weaknesses when
combined:
§ Risk Management focuses exclusively on the future. Anything that occurred in the past is of little interest to
the risk management process, since there is no uncertainty associated with past events. How the program
reached it’s current position is not relevant.
§ Earned Value Management relies on a key assumption that future performance can be predicted based on
past performance. The current EVM method wipes out past variances in the Cumulative to Date and Current
Period reporting data submitted in the IPMR. Only in the Central Repository can the time series of past
performance data be found.
With Earned Value Management, its weakness is assuming the future performance can be predicted from the past
and with Risk Management, its weakness is looking only forward with no informing the future from the past,
connecting the two can create synergy not found in the data produced by the individual processes. This is done by
integrating past performance of Earned Value with the forward projections of Risk Management (both Irreducible
and Reducible uncertainties). The outcome is a foreword projection of a risk informed cost and schedule based on
past performance [9].
2.1 What is Risk and How Does It Impact Earned Value Measures?
Risks are future uncertainties relating to achieving program technical and programmatic
performance goals. Defined by (1) the probability of an undesired event or condition and (2) the
consequences, impact, or severity of the undesired event, were that risk to occur. [1]
In his September 24, 2013, white paper, “Better Buying Power 3.0,” the Department of Defense (DoD) Under
Secretary of Defense for Acquisition, Technology, and Logistics (USD(AT&L)) emphasized his priority to improve
leaders’ ability to understand and mitigate technical risk. He stated, “Most of product development revolves
around understanding and managing risk. Risk management is an endeavor that begins with requirements
formulation and assessment, includes the planning and conducting a risk reduction phase if needed, and strongly
influences the structure of the development and test program. All this is necessary to minimize the likelihood of
program disruption and to maximize the probability of fielding the desired product within reasonable time and
cost.” [1]
Risk management is a key element of a Program Manager’s executive decision-making. DOD risk management is
based on the principles that risk management must be forward-looking, structured, continuous, and informative.
The key to successful risk management is early planning, resourcing, and aggressive execution. [1]
§ Using risk to inform program performance can be treated as a lagging or leading indicator. Lagging indicators
of cumulative CPI and SPI have eliminated the time series and the calculated variances need to provide a
credible forecast of their future behavior.

of 24
§ Risk as a leading indicator can be developed by examining the changes in the reducible and irreducible risks
and re-assessing their impacts of future outcomes.
§ It is this Risk Adjusted Estimate at Completion (EAC) and Estimate to Complete (ETC) that starts the
conversation of program success. Taking the risk-adjusted assessment to the WBS end items and the Work
Packages that produce them is the basis of those decisions.
2.1.1 Informing Program Performance With Risk Informed BCWP
Failure to integrate Risk Management, Cost Risk Analysis, and Earned Value Management
contributes to overruns. The Program Manager is denied clear visibility of quantitative Risk
Management that could increase the probability of mission success – Peter Teets, former Under
Secretary of the Air Force
Risk is the potential for performance shortfalls, which when realized in the future, unfavorably impact program
performance with respect to achieving explicitly, established, and stated requirements for:
§ Cost – are we going to stay on our cost plan?
§ Schedule – does the current schedule represent a credible plan for completing on time in the presence of risk?
§ Technical – are the products being produced going to meet the technical and operational needs – the planned
capabilities – of the user?
A Risk Informed Integrated Program Management Report (IPMR) is a critical success factor for forecasting the cost,
schedule, and technical performance of the program. Research has shown there is four factors that impact
program performance.
1. Size – the diseconomies of scale. Effort versus size does not scale linearly.
2. Uncertainty – the cone of uncertainty for information needed to make programmatic and technical
decisions as the program progresses. Risks and their management. Work effort versus progress
uncertainty.
3. Defects in Technology and Process to Produce Technology – their removal and validation of correction.
4. Human Variation – the naturally occurring productivity of human efforts. Variations in plans and
execution of those plans. Recruiting, retention, availability, and capacity of needed talent.
These factors are templates used to compare what is seen in the program performance assessments versus what
we should expect to see in terms of performance. There are checklists that aide in assessing program performance
based on risk.
§ Size Risk Check List – failure to understand, model, or contain size growth increases risk exposure
̶ Is the program estimated close to its actual size (cost and duration)?
̶ Does the program’s schedule permit completion with the estimated effort? If so, what’s the confidence in
this estimate?
̶ Is the program planned commensurate with its size?
̶ Does the program have appropriate staff specializations for its size?
̶ Does the program have appropriate levels of management for its size?
̶ Does the program have programmatic and technical practices appropriate for its size?
̶ Is the program appropriately addressing the factors that scale disproportionately with size (Process
Maturity, Risk Resolution, Requirements Flexibility, Team Cohesion)?
§ Uncertainty Risk Checklist – failure to reduce uncertainty, increases programmatic and technical risk.
̶ Do the program’s estimates and plans account for the Cone of Uncertainty?

of 24
̶ Where will the program’s challenges come from in terms of the Intellectual Phase Profiles?
̶ Is requirements uncertainty addressed and manageable?
̶ Is design uncertainty addressed and manageable?
̶ Is technology uncertainty addressed and manageable?
̶ Is the degree of BCWS manageable for the size of the program?
̶ Is planning uncertainty addressed and manageable?
̶ Is the program striking an appropriate balance between the time allocated for proactive activities versus
time allocated for reactive activities?
̶ Is risk management in place and appropriate for the size of the program?
̶ Is the overall level of uncertainty manageable for the size of the program?
§ Defect Risk Check List – failure to remove defects increases the risk of product performance failures
̶ Is the program using practices that will minimize the gap between defect insertion and defect detection?
̶ Is the series of defect removal practices capable of producing the desired level of quality?
̶ Is the series of defect removal practices efficient in achieving the desired level of quality?
̶ Are the quantity and kinds of defect removal appropriate for the size of the program?
̶ Are the quantity and kinds of defect removals appropriate for the quantity and kind of uncertainty on the
program?
̶ Are the quantities and kinds of defect removals appropriate for the capabilities of the staff working on the
program?
§ Human Variation Risk Checklist – failure to contain variation of human processes increase risk exposure
̶ Does the staff on the program have the skills to complete a program of the intended size?
̶ Does the staff on the program have the skills to complete a program with this program’s uncertainty
characteristics?
̶ Does the staff on the program have the skills to complete a program with this program’s intended quality
level?
̶ Are the required skill levels matched to both with the size of the program and degree of challenge in the
requirements area?
̶ Are the design/architecture skills level matched to both with the size of the program and degree of
challenge in the design area?
̶ Is the program management skill and experience matched to the program size and overall challenge?
̶ What is the motivation level of the staff on the program?
̶ Does the level of staff turnover support a program of the intended size?
̶ Do staff capabilities support the human/staff organization of the program, including geographic
distribution?
̶ Is the staff’s experience in the business area suitable for the size, uncertainty level, and desired quality
level of the program?
̶ Is the staff’s experience in the technology platform suitable for the size, uncertainty level, and desired
quality level of the program?

of 24
2.1.2 DI-MGMT-81861 Schedule Risk Assessment in §3.7.1.3.6
A schedule risk assessment predicts the probability of program completion on or before the contractual dates. The
Schedule Risk Assessment (SRA) is used by the contractor to perform a probability analysis of key contract
completion dates.
Three-point estimates (minimum, most likely, and maximum remaining duration) are developed for remaining
durations of all tasks/activities consistent with the authorized work. Three-point estimates consist of minimum
remaining duration, most likely remaining duration, and maximum remaining duration.
Activities and Tasks in the following list require individual estimates:
§ Tasks identified as the primary, secondary, and tertiary driving paths - as specified in the CDRL or Government
diction
§ High-risk tasks in the contractors risk management plan.
So if we're going to successfully predict the program completion date, we need more information than just the
duration’s of the work and their sequencing:
§ Who’s doing the work?
§ What’s the efficacy of these work efforts and does it meet the planned efficacy?
§ What are the impediments to this work - risk
§ Is the work producing technically compliant outcomes? If not what’s the impact on cost and schedule
§ What the variances in labor cost, material costs, non-compliant outcomes?
2.1.3 Program Risk Analysis Background
Central to the notion of risk is uncertainty. Traditional risk analysis focuses on Event Based uncertainty - epistemic
uncertainty or reducible uncertainty. Another source of uncertainty and Risk resulting from uncertainty is the
natural variations in work durations, cost of that work or cost of materials, and technical performance. These
natural variations are irreducible, they are Aleatory in nature. Both these uncertainties create risk to schedule
duration, which is the subject of DID-81861’s Schedule Risk Assessment. These Aleatory uncertainties arise from:
§ The level of available and required resources (the topic of the Staffing section of this Guidebook).
§ Trade-offs between resources and time
§ Possible uncertain event (epistemic uncertainty)
§ Causal factors and interdependencies, including common cause factors that impact more than one activity.
These are common in organizational issues.
̶ Lack of previous experience
̶ Use of subjective rather than objective assessments of progress to plan
̶ Uncertainties - biases - in the basis of estimate
Managing uncertainty in programs must go beyond the variability in work and events that create risk and address
the ambiguity and structure of the knowledge of the risks. To do this we need a model of the relationships
between the triggers from the epistemic uncertainties and the natural variances occurring from in the Aleatory
uncertainty.
This Aleatory uncertainty can be determined from Past Performance. This performance can be represented in
Reference Classes. For event based uncertainty - epistemic - is conditionally dependent on assumptions and
condition that may be explicit or implicit. These assumptions and conditions are major sources of uncertainty and
must be addressed and handled explicitly. This is the role of the Risk Register and its connection to the Integrated
Master Schedule (IMS).

of 24
3 Building the Risk Adjusted Performance Measurement Baseline
Starting with a risk adjusted Performance Measurement Baseline, the information needed to
inform program performance using both reducible and irreducible risk. Statusing the reducible risks
at the planned reduction date are similar to a Technical Performance Measure. Irreducible risks
require schedule margin to protect contract events or end item deliverables [11]
DI-MGMT-81861 calls out conducting a Schedule Risk Assessment (SRA). The DOD Earned Value Management
Interpretation Guide [12], Guideline 6 calls for a fully integrated schedule to facilitate the establishment of a valid
Performance Measurement Baseline (PMB). Scheduling authorized work facilitates effective planning, statusing,
and forecasting, which are critical to the success of a program. This is accomplished through a fully networked
Integrated Master Schedule (IMS) and is a foundational component in the establishment of a valid PMB.
EIA-748-C Guideline 6 asks for schedule status no less than monthly to:
§ Identify and assess of actual progress against the plan.
§ Generate of the critical path, near-critical paths, and driving paths.
§ Incorporate and progress of risk management activities and mitigation actions.
The work products produced by Guideline include:
§ Internal schedules and/or IMS
§ Risk/Opportunity Register (evidence of risk mitigation handling plan in the IMS)
§ Schedule Risk Analysis (SRA)
This Guidebook section will address the last bullet in each list above. Other items are important but outside the
scope of this Guidebook.
3.1 Six Steps to Arrive at a Risk Informed Performance Measurement Baseline
Creating a credible Performance Measurement Baseline (PMB) and successfully executing the program using the
PMB, starts by connecting the dots between the processes in Table 1 and the elements in Figure 2. These connections
ensure the program’s Technical Plan and the Programmatic Plan are integrated starting at the Integrated Baseline
Review (IBR). Making these connections starts with the Systems Engineering activities – where the MOEs, MOPs,
TPMs, …ilities, and Risk are identified. The target values from these measures are flowed onto the IMP and IMS
and used to create work activities for reducible risks, and appropriate schedule margins identified through Monte
Carlo Simulations needed for margins to address the irreducible risks. There is existing guidance in place, starting
at the government Program Management Office, down to the contractor level to guide the development of these
connections.

of 24
Table 1 – Six steps to building a risk adjusted Performance Measurement Baseline that includes Risk Reduction activities, Management Reserve,
and Schedule Margin. This work starts by defining a Risk in the Risk Register and representing this Risk in the IMS as either a reducible risk or an
irreducible risk. Each step is performed in the planned order. The success of the risk informed PMB depends on this order being maintained.
Step Outcome of Each Step
❶
Define
WBS
§ Using SOW, SOO, ConOps, and other program documents, develop WBS for system elements and
work processes that produce the program outcomes.
§ Develop WBS Dictionary describing the criteria for the successful delivery of these outcomes.
❷
Build
IMP
§ Develop Integrated Master Plan, showing how each system element in the WBS will move
through the maturation process at each Program Event.
§ Define Measures of Effectiveness (MOE) for each Accomplishment (SA).
§ Define Measures of Performance (MOP) for each Criteria (AC).
❸
Identify
Reducible
Risks
§ For each key system element in the WBS, identify reducible risks, probability of occurrence,
mitigation plan, and residual risk in the Risk Register.
§ Risk mitigation activities will be placed in the Integrated Master Schedule (IMS).
§ For risks without mitigation plans, place budget for risk in Management Reserve (MR) to be used
to handle risk when it becomes an Issue.
❹
Build
IMS
§ Arrange Work Packages and Tasks in a logical network of increasing maturity of system
elements.
§ Define exit criteria for each Work Package to assess planned Physical Percent Complete to
inform BCWP with TPM, MOP, MOE, and Risk Reduction assessments.
❺
Identify
Irreducibl
e Risks
§ For irreducible risks in the IMS, use Reference Classes for Monte Carlo Simulation anchored with
Most Likely duration to calculate needed schedule margin.
§ Assign schedule margin to protect key system elements, per DI-MGMT-81861 guidance.
❻
Baseline
PMB
§ Using risk adjusted IMS containing reducible risk and explicit schedule margin tasks, establish
the Performance Measurement Baseline (PMB)
§ Calculate the Management Reserve (MR) for unmitigated risk from the probabilistic cost, using
Monte Carlo Simulation, and the deterministic cost to the desired confidence level.
§ This Management Reserve is used for unmitigated risks in the risk register.
The risk adjusted PMB receives information about risks from sources show in Figure 2.
It is not the intention of this Guidebook section to show how to build the WBS, IMP, and IMS, but how to risk
inform the PMB, using the Risk Register, the reducible and irreducible uncertainty that creates risk, and the
parameters of the Probability Distribution Functions that describe the behaviors of the uncertainties that create
risk for the program
Figure 2 – Programmatic elements needed to increase the probability of program success. Connecting the dots between Measures of
Effectiveness, Measures of Performance, and Technical Performance Measures needed to inform BCWP that provide leading indicators of
performance needed to take corrective actions to Keep The Program GREEN.

of 24
3.1.1 Risk Management Starts with the Work Breakdown Structure
DODI 5000.01 tells us to start with the Work Breakdown Structure (WBS) for risk identification, the Defense
Acquisition Guide (DAG) also states:
Risk is a measure of future uncertainties in achieving program performance goals and objectives
within defined cost, schedule, and technical and programmatic performance constraints. Risk can
be associated with all aspects of a program (e.g., threat environment, hardware, software, human
interface, technology maturity, supplier capability, design maturation, performance against plan,)
as these aspects relate across the Work Breakdown Structure (WBS) and Integrated Master
Schedule (IMS). [3]
With the guidance shown above, Figure 2 shows WBS as the first place the program to encounter risk management
along with their identification and the description of their outcomes, in the WBS Dictionary.
Each key system element in the WBS must be assessed for technical or programmatic risk that will impede its
delivery on time, on cost, and on performance. The reduction of this risk, following its planned risk reduction
levels, will be used to inform Physical Percent complete and the resulting BCWP. If risk is not being reduced as
planned, the program’s Earned Value measures may not properly represent the technical progress.
3.1.2 Finding Risks in the Integrated Master Plan (IMP)
The IMP is an event-based plan consisting of a sequence of program events, with each event being supported by
specific accomplishments, and each accomplishment associated with specific criteria to be satisfied for its
completion.
Guideline 6 calls out the Integrated Master Plan as a typical work product. With the WBS, the IMP is the next place
for look for risks. Figure 3 shows the structure of the Integrated Master Plan (IMP), with Program Events (PE),
Significant Accomplishments (SA), and Accomplishment Criteria (AC). This structure assesses the increasing
maturity of each key system element, to assure the end item system elements meets the planned Effectiveness
and Performance needs that fulfill the capabilities for the mission or business goals.
With this structure, Earned Value Management measures can now be connected to the Work Packages defined in
the Integrated Master Schedule (IMS) to assess technical performance in ways not available with CPI and SPI only.
The Program Manager now has leading indicators of the program success through the MOEs defined by the SA’s
and MOPs defined by the AC’s, each assessed for compliance with plan at the Program Event. In Figure 2 the IMS
should encompass the IMP; work packages and tasks are the “new” element the IMS adds to the IMP along with
the element of time.

of 24
Figure 3 – The Integrated Master Plan defines the increasing maturity of each key system elements assessed at Program Events. Significant
Accomplishments are entry criteria for this maturity assessment. Accomplishment Criteria are measures substantiating the maturity level of the
work products produced by Work Packages in the IMS.
3.1.3 Creating a Risk Informed PMB from WBS to IMP
Throughout all product definition processes, technical and programmatic risk assessment is performed. These risks
are placed in the Risk Register shown in Figure 3 with their uncertainties. Uncertainty comes in two forms:
Table 2 – Irreducible and Reducible uncertainty both create risk to the program for cost, schedule, and technical performance. As well as
reducible and irreducible there are Unknown Unknowns that are mitigated in the DOD by replanning the program.
Reducible Uncertainty Irreducible Uncertainty
The probability that something will happen to impact cost,
schedule, and technical performance of the system
elements.
The natural variation of the program’s activities work
duration and cost. The variance and its impacts are
protected by schedule and cost margin.
§ Reducible uncertainty can be stated as the probability of
an event, and we can do something about reducing this
probability of occurrence.
§ These are (subjective or probabilistic) uncertainties that
are event-based probabilities, are knowledge-based, and
are reducible by further gathering of knowledge.
§ Irreducibility uncertainty is the probability range of these
variances of the stochastic variability from the natural
randomness of the process and is characterized by a
probability distribution function (PDF) for their range and
frequency, and therefore are irreducible.
§ Irreducible uncertainty can be modeled with Monte Carlo
Simulation using Reference Classes based on past
performance.
3.1.4 Capturing Reducible Risk Starts with the Work Breakdown Structure (WBS)
With Reducible risks, specific work is performed on baseline to reduce the probability of occurrence or impact or
consequence of the risk. With Irreducible risk, schedule margin is needed to protect the delivery date of key
system elements. A measure of the performance of the program is the Margin Burn-down Plan shown in Figure 8.
If the actual risk reduction does not follow the risk burn-down plan, this is a leading indicator of future difficulties
in the program’s performance and an indicator of impact on cost and schedule.

of 24
Figure 4 – The mitigation activities for Reducible Risks are contained in the Risk Register and the WBS in the IMS. Work activities to mitigate
reducible risk is planned and executed On Baseline through Work Packages. Risk reduction lowers the probability and/or impact of occurrence
and lowers the probability of impact of the named risk.
3.2 Steps to Building a Risk Informed Integrated Master Schedule (IMS)
A major source of risk in project management is inaccurate forecasts of project costs, demand, and
other impacts. [14] While it is outside the scope of this Guidebook to show how to develop
accurate and precise estimates there are many sources to guide this effort. [27]
DI-MGMT-81861 calls out in §3.1.7.3, three tiers of the Integrated Master Schedule (IMS) – Summary (§3.1.7.3.2),
Intermediate (§3.1.7.3.3), and Detailed (§3.1.7.3.4). Starting with the WBS, a credible IMS that produces the
system elements must address reducible risk through risk mitigation activities. This IMS must address the
irreducible uncertainties and those discrete risks that remain in the risk register that could not be mitigated
through work activities. These uncertainties are categorized as known–unknowns because they are known, but not
known for certainty they will occur.
Programs can protect against these uncertainties by setting cost and schedule margin for the irreducible
uncertainties, and management reserves for the reducible risks that were not mitigated. Cost and schedule
margins are included in the Performance Measurement Baselines and Management Reserve are established for
the latent reducible risks and is held above the PMBs, but is still part of the Contract Budget Base (CBB).
These three tiers are where risks are located from the WBS and Integrated Master Plan (IMP). Risk mitigation
activities for reducible risks are placed in the IMS. Irreducible risk is handled with Schedule Margin in the IMS.
3.2.1 Reducible Risk Mitigation Processes Are Placed In The IMS
All reducible risks identified in the Risk Register need to be assessed for reducibility. The activities to reduce these
identified risks are to be places in the IMS. If the risk – the uncertainty of an event based outcome – can be
reduced, then that work is assigned to work packages and activities in the IMS and placed on baseline. These risk
buy down activities are managed just like ordinary work, funded by the CBB, measured for performance just like
any other work.
The risk reduction mitigation activities are planned to achieve a specific level of risk reduction at a specific time in
the IMS. Meeting the planned risk reduction level at the planned time is a measure of performance of the risk
retirement activities.

of 24
3.2.2 Irreducible Risk Management Placed In Schedule Margin in the IMS
With the reducible risks from the Risk Register handled with risk retirement activities in the IMS, the irreducible
risks now need to be identified and handled in the IMS. Since the irreducible risks are actually irreducible, only
margin can be used to protect the system elements from this naturally occurring variance. No actual work can be
done to reduce this risk.
Monte Carlo Simulation of the IMS is the primary tool for assessing how much margin is needed for each
irreducible risk type.
3.2.2.1 Development of Schedule Margin For Irreducible Risk
Irreducible uncertainty and the resulting risk can only be handled by schedule margin has a buffer of time used to
increase the probability that the program will meet the targeted delivery date. Schedule Margin is calculated by
starting with a Probability Distribution Function (PDF) of the naturally occurring variance of the work duration of
the Most Likely value of that duration. With the Most Likely Value and the PDF for the probability of other values,
the durations of all work in the Integrated Master Schedule and the probabilistic completion times of this work can
be modeled with a Monte Carlo Simulation tool. This is the basis of the Schedule Risk Assessment in DI-MGMT-
81861 §3.7.1.3.6. While this clause does not call out specific actions, just that a SRA will be performed, this
Guidebook section will show how to calculate the needed schedule margin to protect the key contract
deliverables.
This modeling starts with the deterministic schedule, which includes risk buy down activities for the Reducible
Risks. The Monte Carlo Simulation (MCS) of the deterministic schedule will show the probability of completing on
or before some need date in the future, as shown in Figure 5.
The difference between the deterministic schedule completion date (8/4/1908) and the probabilistic schedule
complete date at some agreed upon confidence level – 80% (8/14/1908) in the example in Figure 5, is the total
schedule margin needed to protect the contractual delivery date. The schedule margin is the difference in the
initial deterministic date and a longer duration and associated date with a higher confidence level generated
through the Monte Carlo Simulation. 1
Figure 5 – Monte Carlo Simulation used to determine schedule margin developed from the Deterministic IMS, showing a
deterministic completion date of 8/4/08. The 80% confidence of completing on or before 8/14/08. The Schedule Margin is then
added in front of key system elements to protect their delivery dates to meet the contractual need date of 8/31/08

of 24
3.2.2.2 Assign Schedule Margin to Protect End Item Deliverables and Contract Events
Schedule Margin shall be placed as the last task/activity/gap before a contract event or end item
deliverable. [11]
When using schedule margin to protect against schedule risk – created by the natural uncertainties in the work
durations enables on-time contractual end item deliveries – there are two schools of thought on how schedule
margin should be managed.
§ Place all schedule margin at the end of the program or system elements. This creates the opportunity to hide
the impact of unplanned schedule margin consumption on intermediate contract events or end item
deliverables
§ Distribute margin at strategic points along critical paths where there are known schedule risks. This is the
preferred way to protect contract events and end item deliverables with explicit schedule margin resulting for
reducible and irreducible risk. As stated in DI-MGMT-81861, this approach is not only allowed, it is preferred
to avoid the issues of placing schedule margin at the end of the program.
In both cases Schedule Margin is not the same as Schedule Slack or Schedule Float. Slack and Float
are Open gaps between the completion of one activity and the start of another. As such this float of
slack is compressible as the activity to the left of the float becomes late the activity to the right of
the float is not impacted until the entire float is consumed. Schedule Margin is a rigid activity that
moves when the activity to the left is delayed; showing explicitly that the Schedule Margin is being
consumed, either as planned or faster than planned.
Placing all the margin at the end appears effective in short duration or production efforts where the primary
schedule risk is not driven by technical complexity. The same objective can be achieved when a disciplined process
is followed for control and consumption of distributed margin. Paramount to this approach is accelerating
downstream efforts when margin is NOT consumed as shown in Figure 6.
Most schedule risk in a development program is encountered when program elements are integrated and tested.
Even when margin is distributed, margin is often kept at the end of the schedule to help protect against risk when
all paths come together during final integration and test. This is the Merge Bias issue with networked activities.
This approach enables on-time end item delivery with realistic cost and schedule baselines that provide accurate
forecasts and decisions based on current status, remaining efforts and related schedule risks.
There are several reasons for distributing schedule margin earlier in the IMS including:
§ Protecting use of critical shared resources so that being a few weeks late doesn’t turn into a several month
schedule impact. An example in space programs is use of a thermal vacuum chamber shared across multiple
programs at critical times in their schedules. If a program is unable to enter the chamber at their scheduled
time the ultimate delay may be an exponential factor of their original delay.
§ Protecting highly visible milestones that are difficult and undesirable to change like a Critical Design Review
(CDR).
§ Establishing realistic performance baselines accounting for schedule risk at key points provides more valid
data to make program decisions.
§ Establishing realistic baselines that are cost effective.
§ Placing margin where we believe it will be needed and consumed provides the most realistic schedule baseline
possible for succeeding efforts and enables more accurate resource planning for prime contract, customer,
and suppliers.

of 24
3.2.2.3 Insertion Points for Schedule Margin
Schedule Margin is a management tool to mitigate the consequences of imperfect planning and
execution. … knowledge is imperfect and highly accurate forecasting, especially in multiyear
projects is impossible. Consequently, schedules account for imperfection through the identification
of risks, schedule risk analysis, and the inclusion of Schedule Margin in the Integrated Master
Schedule to plan for schedule perturbations due to unforeseen, in-scope issues (and naturally
occurring duration variances). [20]
Schedule Margin is not the same as Schedule Slack or Schedule Float as stated in the GAO Schedule Assessment
Guide. 1
Margin is preplanned and consumed for known schedule irreducible uncertainty and float is the calculated
difference between early and late dates. In many ways margin is much like management reserve and float is similar
to underruns/overruns.
Schedule margin is placed where there is known irreducible schedule risk. It is never consumed because of poor
schedule performance. In this case, Schedule Margin is managed like Management Reserve.
Schedule margin is not budgeted – it does not have an assigned BCWS. If the risk the margin is protecting comes
true, new tasks need to be identified and budgeted. If the risk is not realized, the schedule margin is zeroed out
and the succeeding tasks accelerated – moved to the left.
Allocating margin for known risks at key points prevents this margin from being used to cover poor schedule
performance. This forces an immediate recovery action to stay on schedule instead of degrading margin as if it was
schedule float.
Inclusion of margin – either distributed in front of key system elements or at the end of contractual system
elements – does not affect that contractual period of performance. The period of performance is defined by the
contract. The Schedule Margin activities in the deterministic schedule are represented in the PMB, using the task
label defined in DI-MGMT-81861 §3.7.2.4 – SCHEDULE MARGIN.
Inclusion of schedule margin for known schedule risk provides a realistic baseline and accurate resource planning
(including the customer). If not consumed, the effort is accurately represented as “ahead of schedule”.
Schedule margin in an IMS is critical to successfully delivering on time. Without schedule margin, the naturally
occurring variances in work effort will create late conditions on day one, that cannot be recover. One approach is
to hold schedule margin at the end item deliverable. This helps manage the uncertainties and risks of all paths that
land on the contractual end item delivery date. If this is the only place margin is kept, it may cause affordability
issues through unnecessary premium payments and rework.
A more efficient approach is to hold adequate margin in front of end item deliverable as well as placing margin at
key integration and test points. These are high convergence points. When items come together or get tested
together, issues often arise. Placing margin where there is unmitigated risk or where there are likely issues to be
encountered enables the establishment of a realistic baseline and the improved management of risk.
Other key locations for margin include subcontract deliveries or activities where the likelihood and impact of a late
delivery significantly impacts end item delivery or contract events. Alternately, integration points that have
acceptable alternate (later) points of incorporation likely don’t need margin.
1 GAO Schedule Assessment Guide, page 113. “Schedule margin is calculated by performing a schedule risk analysis and comparing the
schedule date with that of the simulation result at the desired level of uncertainty.”

of 24
The critical success factor to schedule margin placement is to assure that the successor tasks can be started if the
distributed margin task is not needed where planned. When this happens, the unused margin can be redistributed
downstream and/or allowed to become additional float.
Figure 6 – Schedule margin protects a deliverable date. When a risk is encountered, alternative plans must also assure the
delivery date can be met. If the schedule margin task is not fully consumed the successor activities must be able to start early.
With this ability unused schedule margin can be accumulated to increase the probability of program success
3.2.3 Development of Cost Margin or Contingency Reserve
The cost margin is the amount of cost reserve needed to address irreducible cost variances of the program’s work
efforts and improve the probability of meeting the target cost – the contract cost. Cost margin is calculated in the
same manner as schedule margin. The contractor develops a probability distribution of the final cost based on the
natural variances contained in the historical databases. The confidence level of meeting the target contracted cost
in the IMS is noted. If the confidence level is too low, is has been suggested that cost margin be added to the
baseline, in the same manner as schedule margin, to bring the baseline up to a desired cost confidence level.
However, at this time, there is no generally agreed mechanism to do so. If the Confidence Level is unacceptably
low, the contractor must redo the IMS and try to reduce costs for time dependent and time independent costs to
raise the cost confidence to an acceptable level. Re-planning would result in re-calculating the schedule margin. If
after re-planning, the cost confidence level is still unacceptably low, the contractor should report the Confidence
Level to the customer at the time of the Integrated Baseline Review (IBR). If the customer agrees the Confidence
Level is too low, resources could be added and the contractor would update the IMS or de-scope the contract to
improve the Confidence Level. Alternatively, the customer would hold the cost margin as a contingency for
overruns.
3.2.4 Development of Management Reserve
Management Reserve is the financial resources needed to address the reducible risks that do not have risk
reduction activities in the IMS. The optimal way to develop management reserves is to re-run the Monte Carlo
Simulation tool with only the these risks against the resource-loaded IMS. The difference between the
deterministic cost estimate and the point on the cost distribution curve for the desired cost confidence level is the
Management Reserve.
3.3 Summary of Establishing a Risk Adjusted PMB
In preparation for program execution, using the activities described above, the Performance Measurement
Baseline can now be prepared for execution. This process includes:
5 Days Margin
5 Days Margin
Plan B
Plan A
Plan B
Plan AFirst Identified Risk Alternative in IMS
Second Identified Risk
Alternative in IMS
3 Days Margin Used
Downstream
Activities shifted to
left 2 days
Duration of Plan B < Plan A + Margin
2 days will be added
to this margin task
to bring schedule
back on track

of 24
§ For each reducible risk, identify the probability of occurrence, the cost and schedule impact, the probability of
those impacts, any mitigation activities, the residual risk remaining after mitigation, and the probability of the
effectiveness of that mitigation.
§ For each irreducible risk, identify the Probability Distribution Function representing the naturally occurring
uncertainties in the modeled variable – duration or cost. Apply this PDF to each work activity impacted by this
risk.
§ Run the Schedule Risk Assessment to determine the variances in cost and schedule using a Monte Carlo
Simulation.
§ From this analysis establish the needed schedule margin, risk burn down plans, and Management Reserve.

of 24
4 Informing Current Period Performance
Integral to establishing the Performance Measurement Baseline (PMB) is the use of an integrated
network schedule (EIA-748-C Guidelines 6 and 7). The guidelines in this category require
development of an integrated network schedule that establishes and maintains a relationship
between technical achievement and progress status. [12]
Information is needed to inform program performance beyond just BCWP based on cost and schedule
performance. This needed information is about how the program is progressing toward delivering the needed
capabilities. How these capabilities are being fulfilled as the program progresses. What risks are bought down at
the planned time to increase the probability of success? How the Technical Performance Measures are being
assessed compared to the planned measures needed to deliver the needed capabilities.
4.1 Connecting the Dots During Execution of the IMS
A comprehensive IMS used to manage the program on a daily basis. It is normally provided by the
contractor via a Contract Data Requirements List (CDRL) item. It is updated on a regular basis. It
should contain all of the contract IMP events, accomplishments, and criteria from contract award to
completion of the contract . . .
Some reminders from DOD guidance before starting to inform program performance with Risk. Without these
connections the Earned Value measures cannot be informed by the technical performance. And without these
connections, the Earned Value measures are simply a reflection of the passage of time and consumption of
resources, with no connection to the planned technical maturity of the program’s system elements.
§ Risk retirement burn down, Figure 8, shows planned burn done of each risk from Red, to Yellow, to Green on
the planned date. The work activities performed to produce this burn down are shown in the IMS. If the risk
reduction did not occur on the planned date, then the program performance measures – BCWP – need to be
informed by this lower than planned performance outcome
§ Schedule margin utilization, Figure 9, shows how schedule margin is consumed as the program progresses. If the
schedule margin is consumed faster than planned, risk to the end item deliverables or key program events
increases. This unfavorable outcome informs program performance measures – BCWP – since schedule
performance and the resulting cost impacts increases.
With the MOEs, MOPs, KPPs, Risks and their reduction plans, and TPMs assigned to the IMP and IMS, we have all
the pieces to connect the dots.

of 24
Figure 7 – starting with the IMP and the measures of progress to plan at the Program Event, Significant Accomplishment, and Accomplishment
Criteria levels, the IMS is constructed and work performed in Work Packages can be assessed as Physical Percent Complete in units of measure
meaningful to the decision makers.
These connections define the needed data for a risk informed PMB. Both reducible and irreducible risks are
required to inform both current period and future period performance based on risk.
Identify these risks starts with the WBS, but is applied the to the Integrated Master Plan’s Program Events (PE),
Significant Accomplishments (SA), and Accomplishment Criteria (AC). Each of these, as well as WBS elements are
sources of risks to be placed in the Risk Register, mitigated in the IMS with Schedule Margin or Risk Buy Down
activities.
4.1.1 Risk Retirement as a Measure of Program Performance
BCWP can be informed with the performance of the planned risk retirement processes on the planned day for the
planned cost, to the planned risk level. The work activities of risk reduction are no different than work activities
needed to produce system elements. They are on baseline, produce outcomes and have technical assessment of
their progress to plan.
Figure 8 – the risk reduction plan for the Center Of Gravity going out of bounds for a flight vehicle as a function of time in the
Integrated Master Schedule. Each numbered item is a risk assessment activity with resulting measures of current risk against
planned risk. Baseline work is performed to reduce this risk and reduce the impact of the risk. Making the reductions to the
planned level on the planned day informs the BCWP of the Earned Value numbers. Reducing the risk late must reduce the
BCWP for activities related to the risk.

of 24
4.1.2 Schedule Margin Burn-Down as a Measure of Program Performance
Informing program performance with Schedule Margin has two components:
§ The amount of schedule margin needed to protect key contract events or end item deliverable as described in
DI-MGMT-81861 §3.7.2.4
§ Any changes in Schedule Margin as described in DI-MGMT-81861 §3.6.7.3
Schedule Margin is a duration buffer prior to a system element delivery date or any contract event. This schedule
margin is a Risk Mitigation process applied as the project progresses, the length of the schedule margin task is re-
evaluated and adjusted as needed to protect the system elements from risks of delay that result from natural
variances in work effort durations.
The Schedule Margin Burn Down shown in Figure 9, shows the use of schedule margin over time. Schedule Margin
buffers work activity duration uncertainties to protect key contract deliverables. As a program progresses, the total
schedule margin is re-evaluated and adjusted to protect the system elements from risks that arise from natural
variances in duration.
When Schedule Margin is consumed faster than planned, this indicates cost and schedule progress is at risk, since
uncertainty has not been controlled as planned. Schedule Margin is both a risk mitigation process and a Risk
Indicator
The identified schedule margin is then be used to inform program performance by comparing planned schedule
margin with actual schedule margin and assessing the impact on the forecast performance.
Figure 9 – Schedule Margin Burn-Down Plan depicts the planned schedule margin versus of the actual schedule margin consumption that
protects the date of a key system elements. Status against this schedule margin plan is a leading indicator of the risk to the delivery date and
the success of the program.
With the Green, Yellow and Red bands defined and the actions to be taken when the schedule margin stays in the
Green or enters the Yellow zone, BCWP can be informed. If the planned schedule margin falls below the planned
margin, then a contingency plan must be implemented. The source of this action can then inform program
performance.

of 24
§ What performance shortfalls occurred that required the use of schedule margin?
§ Where these shortfalls reducible or irreducible?
§ Did the use of schedule margin impact future work and therefore impact future program performance?

of 24
5 Informing Forecast Performance
Any condition threatening the health of program development must be identified sufficiently early
to allow managers to mitigate those areas of technical, cost, and schedule risk. [24]
Informing current period performance with current period risk assessment is straightforward as shown in §4.
Forecasting future performance is also straight forward, assuming the following conditions are in place:
§ A risk adjusted Integrated Master Schedule with reducible and irreducible risks assigned to work.
§ A risk register in in place describing the reducible or irreducible risks and the their periods of performance
o Reducible risks have mitigations assigned for their period of performance
o Reducible risks without mitigations have Management Reserve with sufficient funding to cover the
probabilistic occurrence of the risk
o Irreducible risks have Reference Classes Probability Distribution Functions for task durations are
available for future work.
Using the risk adjusted Performance Measure Baseline and the current period performance, informing program
performance in the future has straightforward steps.
At the end of each current period reporting:
§ Assess changes to reducible risks, their probability of occurrence, cost or schedule impact, mitigations, and
residual risk for each risk in the Risk Register for future (past time now) probabilistic event.
§ Assess changes to irreducible risks, changes to the Probability Distribution Function assigned to the risk, and
the PDF assignment to each work activity in the IMS with it’s period of performance
§ Reapply the Monte Carlo Simulation with this updated information to produce a new probability of cost and
schedule distributions
§ With this information assess the Management Reserve for schedule and cost compared to the Baseline cost
and schedule to determine sufficient reserves.
§ If there are not sufficient reserves, make adjustments to the logic of the IMS, work efforts to adjust work
durations.
After adjustments to the reducible and irreducible risks the outcome of the Schedule Risk Assessment will show
the confidence of meeting cost and schedule goals. A typical chart is shown in
Figure 10 – a typical cost and schedule probabilistic assessment of future program performance based on past performance.
The probability of on or before a date and at or below a budget informs the forecast performance of the program.

of 24
6 References
[1] DOD Risk Management Guide V7, http://www.acq.osd.mil/se/docs/DoD-Risk-Mgt-Guide-v7-interim-
Dec2014.pdf
[2] Integrated Master Plan and Integrated Master Schedule Preparation and Use Guide,
http://www.acq.osd.mil/se/docs/IMP_IMS_Guide_v9.pdf
[3] Defense Acquisition Guidebook (DAG), https://dag.dau.mil/Pages/Default.aspx
[4] Project Risk Management: Process, Techniques, and Insights, 2nd
Edition, Chris Chapman and Stephen Ward,
John Wiley & Sons, 2003.
[5] Technical Risk Management, Jack V. Michaels, 1996, Prentice-Hall
[6] Effective Opportunity Management of Projects: Exploiting Positive Risk, David Hillson, 2004, Taylor & Francis
[7] Effective Risk Management: Some Keys to Success, 2nd
Edition, AIAA Press, 2003.
[8] Integrating Risk Management with Earned Value Management,
http://www.ndia.org/Divisions/Divisions/Procurement/Documents/Content/ContentGroups/Divisions1/Proc
urement/Integrating_RM_with_EVM.pdf
[9] “Combining Earned Value Management and Risk Management to Create Synergy,” Dr. David Hillson,
http://risk-doctor.com/pdf-files/cev-a1004.pdf
[10] Interfacing Risk and Earned Value Management, Association for Project Management, 2008.
https://pbcdata.files.wordpress.com/2011/03/interfacing-risk-and-earned-value-management.pdf
[11] DI-MGMT-81861 Integrated Program Management Report (IPMR)
[12] Department of Defense Earned Value Management Interpretation Guide, OUSD AT&L (PARCA), February 18,
2015.
[13] Taxonomy-Based Risk Identification, Technical Report, CMU/SEI-93-TR-6, ESC-TR-93-183,
http://www.sei.cmu.edu/reports/93tr006.pdf
[14] From Nobel Prize to Project Management: Getting Risks Right, Bent Flyvbjerg, University of Oxford, Said
Business School, April 1, 2006, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2238013
[15] A Guide to Managing Programs Using Predictive Measures,
http://www.ndia.org/Divisions/Divisions/IPMD/Documents/WorkingGroups/Predictive_Measures_Guide_IP
MD_Review_Copy.pdf
[16] Cost Risk and Uncertainty Analysis Guidebook, https://acc.dau.mil/adl/en-
US/316093/file/46243/AF_Cost_Risk_and_Uncertainty_Guidebook_Jul07.pdf
[17] LANL’s Development of Schedule Contingency Based on Probabilistic Risk Results, Proceedings of the Project
Management Institute Annual Seminars & Symposium November 1–10, 2001, Nashville, Tennessee

of 24
[18] Integration Of Risk And Opportunity Thinking In Projects, Kalle Kähkönen, VTT Building and Transport,
Finland, Presented at the Fourth European Project Management Conference, PMI Europe 2001, London UK,
6-7, June 2001.
[19] Probabilistic Schedule Reserve Allocation,
http://www.nasa.gov/sites/default/files/files/Probabilistic_Schedule_Allocation_2013_TAGGED.pdf
[20] Depicting Schedule Margin in Integrated Master Schedules, Depicting Schedule Margin in Integrated Master
Schedules,
http://www.ndia.org/Divisions/Divisions/IPMD/Documents/WhitePapers/NDIAScheduleMarginWhitePaperFi
nal-2010%282%29.pdf
[21] Integrated Master Plan and Integrated Master Schedule Preparation and Use Guide, The Office of the
Secretary of Defense (OSD) Acquisition, Technology, & Logistics (AT&L),
www.acq.osd.mil/se/docs/IMP_IMS_Guide_v9.pdf
[22] “Prediction of project outcomes: The application of statistical methods to Earned Value Management and
Earned Schedule Performance Indexed,” Walt Lipke, Ofer Zwikal, Kym Henderson, and Frank Anbari,
International Journal of Project Management, 27 (2009), 400-407.
[23] Integrating Risk Management with Earned Value Management, NDIA,
http://www.ndia.org/Divisions/Divisions/Procurement/Documents/Content/ContentGroups/Divisions1/Proc
urement/Integrating_RM_with_EVM.pdf
[24] “Technical Performance Measurement, Earned Value, and Risk Management: An Integrated Diagnostic Tool
for Program Management,” Commander N. D. Pisano, SC, USN, Program Executive Office for Air ASW,
Assault, and Special Mission Programs (PEO(A)), www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA404646
[25] “Monitoring Risk Response Actions for Effective Project Risk Management,” Edouard Kujawski and Diana
Angelis, Systems Engineering, Vol. 13, No. 4, 2010
[26] Transforming project risk management into project uncertainty management Stephen Ward and Chris
Chapman, International Journal of Project Management, 21 (2003) 97–105,
http://web.nchu.edu.tw/pweb/users/arborfish/lesson/10490.pdf
[27] GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program
Costs, March 2009, GAO-09-3SP, www.gao.gov/assets/600/591240.pdf
[28] NNSA Cost Estimating Guide, 50.005, http://www.efcog.org/wg/pm_ce/docs/HQ-406529-v1-
Signed_NNSA_Cost_Estimating_Guide_50_005.pdf
[29] A Guide to Managing Programs Using Predictive Measures, September 17, 2014, National Defense Industry
Association, Integrated Program Management Division,
http://www.ndia.org/Divisions/Divisions/IPMD/Documents/ComplementsANSI/NDIA_IPMD_Predictive_Mea
sures_Guide_Sept172014(a).pdf

Informing Program Performance with Programmatic and Technical Risk

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Informing Program Performance with Programmatic and Technical Risk

Similar to Informing Program Performance with Programmatic and Technical Risk (20)

More from Glen Alleman

More from Glen Alleman (20)

Recently uploaded

Recently uploaded (20)

Informing Program Performance with Programmatic and Technical Risk