3. 2
Omnia Paratus
Corporation
A recent poll showed that 68% of companies experienced financial
losses directly related to Supply Chain disruptions
Most of the financial impacts were related to supplier performance that did not meet demand
requirements, and delayed, damaged or misdirected shipments.
The majority of companies polled are in the early stages or have yet to think about integrating Risk
Management into their Supply Chain
Supplier related risks are most often identified after contract award is a program issue, not a risk:
– Supplier shipment delay
– Supplier capacity exceeded
– Unable to meet technical requirements
– 1st Article or Flight Test failures
– Parts damaged during shipment or rejected during quality inspection
Schedule delays caused by supplier performance can have an equal or greater financial impact to
a program’s bottom line, but are often overshadowed budget impacts due to supplier cost overruns
4. 3
Omnia Paratus
Corporation
Procurement analysis exists in varying degrees, although the most
commonly used practice is Best Value Analysis (BVA)
Incumbent
Supplier w/ Existing
Capabilities
New
Supplier
Score based on
savings to program
~$300K
Technical Score based on:
- New Capability to Supplier
- Technical deviations req’d
based on proposal
- 1st Article & Flight Req’d
based on Customer reqt’s
BVA considers not only cost, but other quantifiable and non-quantifiable factors supporting an
investment decision
– Utilizes weighting scales for analyzing “True” program value of supplier bids
– Can include, but is not limited to, performance, producibility, reliability, maintainability, and
supportability enhancements
– Intended to select the source offering the greatest overall benefit in response to the requirement
5. 4
Omnia Paratus
Corporation
While a BVA attempts to provide insight to the least risky procurement,
it does not specifically address or integrate impacts of risk
Technical Deviations
required by Supplier
Design changes resulting from
Technical deviations resulted in
cost growth to original
purchase order
Program Award fee lost
resulting in Supplier
delays cause IMS
milestones to be missed
Orders placed were
against original design
and fulfilled by
Incumbent suppler
Impact from production
delays of 4 weeks
Fees required to get
Incumbent supplier
operations back up and
minimize delay of
deliveries to Customer
$390K overrun was a direct
loss of company profit
Performing a sample analysis of a new supplier shows the potential risks that may be incurred when
basing decisions on costs alone
– Technical requirements outside current capabilities
– Schedule impact due from potential delay of 1st Article Testing or Flight Test Requirements
– Cost growth due to technical deviation’s required
– Impact to Operations due to late supplier deliveries
6. 5
Omnia Paratus
Corporation
Despite the value-add it offers to improving the acquisition process,
risk management is seldom considered or implemented
Risk Management Is Not Difficult, Is Well Documented, and Training
Materials Are Readily Available. So Why Is It “Hard”?
– Seldom seems urgent
It’s a “lower right quadrant” activity
It’s often overcome by events
It’s someone else’s job
– Requires careful thought
People think it’s “easy” because it’s not difficult
Fail to distinguish between perception and reality
Skip the analysis and solve the wrong problem
Determining what can be controlled, influenced, or changed
– Team participation
Part of the culture
Common understanding
Training, support, and reinforcement
Risk
Management
Low High
LowHigh
Importance
Urgency
7. 6
Omnia Paratus
Corporation
Risks that impact large scale programs often directly originate from
supplier performance throughout all phases of a program
The main challenges that confront these programs include:
– Program Requirements – Technical requirements can have a tendency to increase unknown
potential for inhibiting program success or realization of full award fee are often over looked
while developing scope of work and supplier selection process.
– Source Selection Analysis - A review of industry standard Best Value Analysis or Lowest
Cost Alternative approach reveals gaps in several areas of Procurement Analysis skewing the
perception of results derived, inherently selecting a supplier who may not adequately meet
program requirements.
– Risk Identification – Most risks identified often have root causes stemming from supplier
performance and/or capabilities, these risks tend to be identified after procurement award.
– Risk Impact Analysis – Procurement analysis does not provide insight into the potential cost
and schedule impacts associated with selecting a supplier, impacts that could affect the
success or ability of program operations, award fee and sustaining customer contracts.
– Mitigation Analysis – Developing a mitigation plan after a problem has occurred limits a
program’s mitigation options resulting in exuberant mitigation costs that extend beyond
program office and/or client budget.
8. 7
Omnia Paratus
Corporation
Request for Proposal
Development
Proposal Evaluation Contract Award
Integrating Risk Management directly into the acquisition analysis
process is seamless and beneficial
Develop and issue RFPs based
on a standard format; collect
vendor proposals
Description
Evaluate proposals using a
clear and structured evaluation
mechanism and methodology
Activities
Assess program requirements
Assess supplier capabilities
Assessment methodology &
criteria development
Define risk parameters
RFP responses collection
Technical and contractual /
procedural evaluation
Risk identification
Risk ratings defined
Supplier proposal risk analysis
Identify potential mitigation plans
Assess mitigation posture
Integration of risk impact into
program budget and schedule
Outcome
Detailed functional and technical
requirements
Evaluation methodology
Scoring model for RFP evaluation
Vendor bidders list
RFP document
Risk rating scales & categories
Technical, contractual /
procedural and commercial
evaluation of proposals
List of potential risks and
risk ratings within individual
proposals
Pre-Mitigation Analysis
Post-Mitigation Analysis and
Effectiveness
Program level risk adjusted
cost and schedule analysis
Supplier selection
Contract Award
Analyze potential risk impact
and mitigation posture relating
to supplier proposal’s
Assess program requirements,
supplier capabilities and evaluation
criteria to establish RFP
Provide insight into where supplier
risks affect the program and uncover
their true impacts.
Compare supplier risk profile at
proposal evaluation completion to
determine outstanding risk
exposure.
9. 8
Omnia Paratus
Corporation
Acquisition Risk Analysis follows a typical Risk Management process
and can be tailored to program specific needs
Risk
Identificat
ion
Sets risk parameters for
effective risk analysis
Examines risks to
determine impacts by
and across alternatives
Identifies risks that
may impact cost,
schedule, or
performance
Develops measures for
keeping risk at
acceptable levels
Uses results of risk
analysis in assessing
alternatives
Risk
Planning
• Collecting SME input,
best practices,
metrics, etc., to
identify possible risks
• Documenting risk
data into a central
repository
1 Risk
Identification
• Creating standard risk
terms and definitions
• Developing processes,
criteria, and scoring
approach for risk
Risk
Analysis
• Conduct probabilistic
assessments of risk
impacts
Risk
Mitigation
• Identifying, evaluating,
and selecting
strategies to reduce
risk
• Developing an
actionable risk
mitigation plan, with
sound rationale (if
applicable)
Applying
Risk Results
• Using cumulative risk
scores as a vehicle for
ranking alternatives
2 3 4 5
10. 9
Omnia Paratus
Corporation
Building qualitative definitions for risk ratings enables an objective, not
subjective quantification of proposal risks
The first step in Risk Planning involves defining standard risk terminology using many different resources, but
should be robust and accurate enough to reflect the program’s overall risk tolerance
Developing a robust, well-defined risk analysis process will produce results that reflect program needs; effectively
communicates these needs across stakeholders; reduce personal bias in determining relative risk importance; and
uncover potential impacts of great importance to the program
Risk criteria is not normative, and are based on available resources, time constraints, and amount/type of
information solicited in generating the criteria
Performance
Metrics
The analyst needs to ensure that the units of measure for risk impact reflect program needs. Critical program drivers might include life-cycle
costs or metrics reliability (e.g., Mean Time Between Failure).
Order of Magnitude
If impact thresholds are expressed in percentages (for example) the analyst needs to review those figures within the context of the overall
anticipated program cost—e.g., 10% of $5 million program is substantially different than 10% of a $5 billion program.
Level of Impact
The analyst needs to ensure that there is an adequate number of thresholds for evaluating the risk impact. The more levels of consequence
that are utilized, the greater the insight into the need for increased data fidelity/quantity of data the analyst would need to consider.
Qualitative vs.
Quantitative
Qualitative data is represented by probability values ranging from “Very Unlikely” to “Very Likely” , using stakeholder input to assess and weigh
the importance of benefits, cost, and risk in relationship to the total analysis or evaluation of benefit, risk, and cost factors that cannot be
quantified.
Quantitative data is represented by consequence values using linear numerical probabilities (e.g., 0.1, 0.3, 0.5, 0.7, and 0.9), nonlinear
numerical probabilities (e.g., 0.05, 0.1, 0.2, 0.4, 0.8), cost estimates, metrics, ranges, or percentages.
Considerations for Defining Risk Criteria
11. 10
Omnia Paratus
Corporation
A Supplier Management Maturity module is used to determine if
supplier capabilities comply with program requirements
Framework used to assess each IPTs risk
management maturity in terms of People, Process,
Technology, and Governance
The optimal goal of the risk enhancement effort is
to select supplier cable of bidding based on their
“Best in Class” maturity
Regular training conducted to enhance
skills and capabilities
Dedicated organizational resources
All staff is informed and capable of
applying mid- to advanced concepts
Applied use of specific
processes/tools
Dedicated team resources
In-house core experts, formally
trained in basic skills
Dedicated resources do not
exist
Limited to individuals who may
have had little or no formal
training
Minimal understanding or
experience in applying basic
concepts and principles
Minimal understanding of
principles or language
Integral to informed decision-making by
upper management
Active use encouraged and rewarded
Part of the organizational philosophy to
achieving program success
Top-down commitment by leadership
• Accepted as a program
management function
Benefits recognized and expected
Upper management requires
tracking and reporting
Focus on mitigation effectiveness
versus reporting and status
tracking
Upper management
encourages, but does not
require use
Application varied through out
program
• Process may be viewed as
additional overhead with
variable benefits
• Minimal awareness
Minimal upper management
involvement
Tendency to continue with
existing processes even in the
face of potential failure
Dedicated resources do not exist
State-of-the-art tools and
methodologies
Distributed data environment that
provides access to all program
resources
Standardized and automated reporting
capabilities
• Integrated set of tools and
methodologies
Centralized data environment
managed by dedicated team
resources
• Customizable solutions tailored
to program
• Few repeatable technological
solutions in place
• No structured application
Management and tracking tools
not in use
Analysis not performed
Qualitative/quantitative analysis
methodologies employed with emphasis on
valid and reliable data sources
Metrics used and reported, with
consistent feedback for improvement
External stakeholders actively
participate in process
Integration into organizational processes
and decision-making
Formal processes integrated into
different areas of the program
Active allocation and
management of budgets
Metrics collected
Key internal stakeholders actively
participate in process
Common processes defined
and formally documented
Process effectiveness limited to
a dedicated team
Qualitative analysis based on
ill-defined rating system
Established decision-body
forum
Inconsistent application of
concepts or principles
Formal risk decision-making
body does not exist
Risk reporting and/or metrics is
minimal or does not exist
Formal, documented process
does not exist
4 – Best in Class3 – High Performance2 – Functional1 – Minimal Capability
PeopleProcessTechnologyGovernance
NOTIONAL
People Process Tech Gov’t Metrics Quality Capacity
Average
Rating
3 4 4 1 2 2 1 2.42
2 3 - 3 3 1 1 1.85Supplier B
3 3 4 2 3 3 2 2.85Supplier C
3 3 4 4 3 3 2 3.14Supplier D
Supplier A
Supplier’s will be scored based on the
maturity model to establish a list of
qualified suppliers for proposal
solicitation
NOTIONAL
12. 11
Omnia Paratus
Corporation
Once proposals have been received a technical evaluation should be
conducted for clarifications prior to identifying supplier risks
Proposal Clarification Items
Topic
General
Area
Staffing
Questions
Is the team available as indicated in the Proposal?
Additionally, a further elaboration on local staff/ qualifications is
required
Ref.
P. 60
Approach
A further elaboration on the timeline is required, especially on the 9
days timeframe proposed for defining the architecture and standards
A show-case of the deliverables must be provided
How realistic are the assumptions on available data and
Organization staff?
P. 46
Assumptions What is the impact on costs and timeline of a provision of a
documentation in other languages?
P. 18
Phase 0: Kick-Off
Phase 1: Baseline
Phase 2: Best
Practice Review
Alignment
Tools
Approach
How will Bidder ensure the alignment of the PM tools to the
Organization’s PMO?
How will the automatic “systems” feed work (XML). Is it a
requirement?
How will any incompatibilities of import functions affect the
suggested timeline?
What if the Organization wants to use other EA tools, like ARIS?
Further elaboration on “GEAS” layers required. A mapping to the
Organization framework layers is needed as well
Explanation of the rationale for selecting Singapore, Australia and
Canada
Is the access to those countries only via databases/benchmarks or
“real” contact?
P. 34
P. 36-
38
P. 43
Phase 3: Definition
of Architecture &
Standards
Imperatives
No reference given, on how the Organization’s imperatives will be
ensure – therefore, a further elaboration on the approach is required
P. 45
Answer Given
Issue
Resolved
All questions to a
Bidder should be
compiled and sent
in a formal email /
fax
All Bidders should
be given ample and
the same amount of
time to respond to
clarifications
requested
13. 12
Omnia Paratus
Corporation
The next step to the risk assessment process is to identify risks that
may impact a program’s cost, schedule, or technical performance……
Risk
Identificat
ion
The risk analysis focuses on risks that affect each supplier’s ability to ensure program success (i.e.,
ongoing, uninterrupted support to the operational forces).
Although identification of risk relies on the skill, experience, and insight of subject matter experts and
risk personnel, the methods and tools for initiating the identification of risk may vary
As such utilizing risk categories such as these bolsters risk identification…
“Common” Risks
Scope of Work
Testing &
Integration
Material
Availability
Suppler Metrics Schedule
Budget
Constraints
Qualification
Requirements
Supplier
Capabilities
Performance
Metrics
Resources
Lessons Learned
ManagementComponents
Program
Requirements
Sensitivity
Analysis
14. 13
Omnia Paratus
Corporation
Technical
…each risk is then reviewed against a set of Program-specific impact
definitions, with the highest rating used to determine overall risk severity
Cost ( % over of cost
target)
Supplier Delivery Schedule
Existing technology does not exists
Existing technology exists, but has not been proven
Supplier has never built component before, Flight Test required
Supplier has never built component before, 1st article required
Supplier has never built component before
Supplier built similar component, Flight Test required
Supplier has built a component similar in nature, 1st article required
Supplier has built identical component for other Programs
Supplier has previously built component
Incumbent supplier
Level
Disastrous
Severe
Critical
Substantial
Significant
High
Moderate
Medium
Low
Minimal
Technical = High 3 Schedule = Critical 1 Cost = High 2
Consequence Rating = Critical
> 6.00%
5.01 – 6.00%
4.01 – 5.00%
3.01 – 4.00%
2.01 – 3.00%
1.01 – 2.00%
.76 – 1.00%
.51 - .75%
.26 - .50%
< .25%
>5 month slip in MRP requirements
4-5 month slip in MRP requirements
3-4 month slip in MRP requirements
2-3 month slip in MRP requirements
1-2 month slip in MRP requirements
< 1 month slip in MRP requirements
Risk erodes 100% of schedule margin
Risk erodes 51 – 75% of schedule margin
Risk erodes 26 – 50% of schedule margin
Risk erodes < 25% of schedule margin
15. 14
Omnia Paratus
Corporation
Risk
#
WBS
Impact
IMS
Task ID
Risk Description
Risk
Rating
Cost Impact
($K's)
Schedule
Impact
(Days)
1 1.2.1.3
53
Lack of supplier staff to support delivery
requirements
3 8 31.00
$7,500 88
2 2.3.1.6 71 CCB is a development item 5 9 71.25 $8,500 118
3 4.3.5.1.1
101
Supplier X's On Time Delivery rating
subpar
2 10 28.50
$9,500 150
4 6.3.3.7
88,97
Requirements of flux capacitor increase
the need for Supplier X System Test &
Eval staff and resources
4 6 32.25
$5,500 52
5 1.3.1.5
70
Lack of technical capability in power
generation / storage hardware may cost /
schedule over runs
1 3 3.30
$2,500 18
6
1.3.1.2
1.3.3.15 67,97
Tooling re-use approach may not be
compatible with new technology
4 10 71.25
$9,500 150
Prob. Conseq.
Risks are then integrated into the suppliers proposal to determine
impacts of risks based on the risk ratings identified
The risk database transposes risk rating into cost and schedule impacts in quantifiable dollar and
days
WBS and IMS task are later used to integrate supplier risk adjusted proposal into the Program’s
Budget and IMS
16. 15
Omnia Paratus
Corporation
Monte Carlo Analysis is used in calculating risks associated with
supplier cost impacts to program budget….
This chart example utilized Crystal
Ball to perform Monte Carlo, multiple
types of software exists and can be
used for this analysis
Pick a Confidence Level based on
program maturity and requirements
This example highlights the 75th
Percentile
– 75% of costs are below the line,
25% of costs are above
– The 75% CL is $49,448
17. 16
Omnia Paratus
Corporation
….as well as supplier risks that have the potential for disrupting a
program’s Operations schedule.
The analysis example utilized
SCRAM to run the schedule
risks analysis
Confidence Level (CL) picked
for cost is also used for
schedule to determine how
risk may impact a supplier’s
delivery schedule
18. 17
Omnia Paratus
Corporation
Schedule & Cost Risk Assessment Module (SCRAM) is a MS
Project add-in and FREE for use on NASA projects.
SCRAM’s capabilities
compare with that of
Pertmaster, @Risk and
Risk++
Extremely user friendly and
reliable
Customizable aspects not
available with other tools
Compatible will all MS Office
Products
19. 18
Omnia Paratus
Corporation
Once Monte Carlo simulations are complete and confidence levels
selected, supplier bids can be compared based on potential risk impacts
Proposed $ 494,958 $ 465,135 $ 514,453
10% $ 507,645 $ 512,622 $ 547,565
20% $ 509,542 $ 523,002 $ 555,345
30% $ 510,834 $ 530,892 $ 561,676
40% $ 512,016 $ 537,855 $ 567,689
50% $ 513,090 $ 544,29 $ 573,385
60% $ 514,241 $ 551,640 $ 579,471
70% $ 517,449 $ 552,673 $ 585,912
80% $ 520,923 $ 565,114 $ 594,321
90% $ 523,007 $ 582,310 $ 607,304
100% $ 629,484 $ 589,685 $ 711,107
Supplier A Supplier B Supplier C
$700,000,000$600,000,000$500,000,000
Confidence Level Chosen
Supplier B
Supplier A
Supplier C
21. 20
Omnia Paratus
Corporation
0
1
2
3
4
5
6
7
8
6/2/10 6/6/10 6/10/10 6/14/10 6/18/10 6/22/10
Material Resource Planning requirement for this component is 04/25/2010
Risk adjusted delivery schedules are then compared to determine
the potential risk impact to program’s operations
Supplier’s Initial Proposal
Supplier A B C
Proposed Schedule 04/18/2010 04/04/2010 04/21/2010
Risk Adjusted Schedule 05/13/2010 07/18/2012 08/21/2012
0
1
2
3
4
5
6
7
8
9
5/3/10 5/7/10 5/11/10 5/15/10
0
0.5
1
.5
2
.5
3
.5
4
4.5
7/7/10 7/11/10 7/15/10 7/19/10 7/23/10 7/27/10 7/31/10 8/4/10 8/8/10
Illustrative IllustrativeIllustrative
Supplier A Supplier B Supplier C
22. 21
Omnia Paratus
Corporation
The risk-adjusted cost and schedule results are then compared for
awarding contracts on a risk averse path
Initial review of Supplier bids would indicate “C” as the supplier of choice
Based on Supplier B & C delivery metrics and potential risk, schedule impact could result in
more than a 3 months past MRP requirements
Based on supplier C’s lack of technical capabilities and schedule risk to operations, risk impact
could equate to ~$100K over component budget
Based on risk adjusted Cost & Schedule proposal analysis supplier “A” should receive program
consensus for contract award based on least amount of risk exposure to the program
Supplier’s Risk Adjusted Proposal
Supplier A B C
Risk Adjusted Cost $ 523,007 $ 582,310 $ 607,304
Risk Adjusted
Schedule
05/13/2010 07/18/2012 08/21/2012
Supplier’s Initial Proposal
Supplier A B C
Proposed Cost $ 497,042 $ 482,331 $ 427,436
Proposed Schedule 04/18/2010 04/04/2010 04/21/2010
23. 22
Omnia Paratus
Corporation
Given these potential benefits, a few key considerations are worth
noting
In order to have a successful portfolio risk management process, it’s important that the
constituent components of the program have sufficiently mature supply chain management
and risk management processes.
Integration of acquisition risk analysis into a program’s budget and schedule is necessary to
capture the magnitude of potential program risk impact by a single supplier
Identifying risks within a proposal enables forward looking program management that can be
streamlined into existing risk database’s for future risk management planning and mitigation.
Qualitative risk analysis provides enhanced proposal evidentiary support and solid justification
for awarding contracts
The success of a supply chain risk management program requires the consistent and active
support of program leadership in order to be successful.
24. 23
Omnia Paratus
Corporation
For more information on how acquisition risk analysis can be
applied to your specific challenges, please:
Contact:
James Taylor
Huntsville, Alabama
310-462-6878
James.Taylor@omniaparatus.com
