Your SlideShare is downloading. ×
  • Like
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply




Published in Business , Economy & Finance
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On SlideShare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. 0 Getting What you Paid for: Acquisition Risk Analysis James Taylor Omnia Paratus Corporation Used with Permission
  • 2. 1 Omnia Paratus Corporation Lamborghini Reventon $1.6M Kia Rio $12,145
  • 3. 2 Omnia Paratus Corporation A recent poll showed that 68% of companies experienced financial losses directly related to Supply Chain disruptions  Most of the financial impacts were related to supplier performance that did not meet demand requirements, and delayed, damaged or misdirected shipments.  The majority of companies polled are in the early stages or have yet to think about integrating Risk Management into their Supply Chain  Supplier related risks are most often identified after contract award is a program issue, not a risk: – Supplier shipment delay – Supplier capacity exceeded – Unable to meet technical requirements – 1st Article or Flight Test failures – Parts damaged during shipment or rejected during quality inspection  Schedule delays caused by supplier performance can have an equal or greater financial impact to a program’s bottom line, but are often overshadowed budget impacts due to supplier cost overruns
  • 4. 3 Omnia Paratus Corporation Procurement analysis exists in varying degrees, although the most commonly used practice is Best Value Analysis (BVA) Incumbent Supplier w/ Existing Capabilities New Supplier Score based on savings to program ~$300K Technical Score based on: - New Capability to Supplier - Technical deviations req’d based on proposal - 1st Article & Flight Req’d based on Customer reqt’s  BVA considers not only cost, but other quantifiable and non-quantifiable factors supporting an investment decision – Utilizes weighting scales for analyzing “True” program value of supplier bids – Can include, but is not limited to, performance, producibility, reliability, maintainability, and supportability enhancements – Intended to select the source offering the greatest overall benefit in response to the requirement
  • 5. 4 Omnia Paratus Corporation While a BVA attempts to provide insight to the least risky procurement, it does not specifically address or integrate impacts of risk Technical Deviations required by Supplier Design changes resulting from Technical deviations resulted in cost growth to original purchase order Program Award fee lost resulting in Supplier delays cause IMS milestones to be missed Orders placed were against original design and fulfilled by Incumbent suppler Impact from production delays of 4 weeks Fees required to get Incumbent supplier operations back up and minimize delay of deliveries to Customer $390K overrun was a direct loss of company profit  Performing a sample analysis of a new supplier shows the potential risks that may be incurred when basing decisions on costs alone – Technical requirements outside current capabilities – Schedule impact due from potential delay of 1st Article Testing or Flight Test Requirements – Cost growth due to technical deviation’s required – Impact to Operations due to late supplier deliveries
  • 6. 5 Omnia Paratus Corporation Despite the value-add it offers to improving the acquisition process, risk management is seldom considered or implemented  Risk Management Is Not Difficult, Is Well Documented, and Training Materials Are Readily Available. So Why Is It “Hard”? – Seldom seems urgent  It’s a “lower right quadrant” activity  It’s often overcome by events  It’s someone else’s job – Requires careful thought  People think it’s “easy” because it’s not difficult  Fail to distinguish between perception and reality  Skip the analysis and solve the wrong problem  Determining what can be controlled, influenced, or changed – Team participation  Part of the culture  Common understanding  Training, support, and reinforcement Risk Management Low High LowHigh Importance Urgency
  • 7. 6 Omnia Paratus Corporation Risks that impact large scale programs often directly originate from supplier performance throughout all phases of a program  The main challenges that confront these programs include: – Program Requirements – Technical requirements can have a tendency to increase unknown potential for inhibiting program success or realization of full award fee are often over looked while developing scope of work and supplier selection process. – Source Selection Analysis - A review of industry standard Best Value Analysis or Lowest Cost Alternative approach reveals gaps in several areas of Procurement Analysis skewing the perception of results derived, inherently selecting a supplier who may not adequately meet program requirements. – Risk Identification – Most risks identified often have root causes stemming from supplier performance and/or capabilities, these risks tend to be identified after procurement award. – Risk Impact Analysis – Procurement analysis does not provide insight into the potential cost and schedule impacts associated with selecting a supplier, impacts that could affect the success or ability of program operations, award fee and sustaining customer contracts. – Mitigation Analysis – Developing a mitigation plan after a problem has occurred limits a program’s mitigation options resulting in exuberant mitigation costs that extend beyond program office and/or client budget.
  • 8. 7 Omnia Paratus Corporation Request for Proposal Development Proposal Evaluation Contract Award Integrating Risk Management directly into the acquisition analysis process is seamless and beneficial  Develop and issue RFPs based on a standard format; collect vendor proposals Description  Evaluate proposals using a clear and structured evaluation mechanism and methodology Activities  Assess program requirements  Assess supplier capabilities  Assessment methodology & criteria development  Define risk parameters  RFP responses collection  Technical and contractual / procedural evaluation  Risk identification  Risk ratings defined  Supplier proposal risk analysis  Identify potential mitigation plans  Assess mitigation posture  Integration of risk impact into program budget and schedule Outcome  Detailed functional and technical requirements  Evaluation methodology  Scoring model for RFP evaluation  Vendor bidders list  RFP document  Risk rating scales & categories  Technical, contractual / procedural and commercial evaluation of proposals  List of potential risks and risk ratings within individual proposals  Pre-Mitigation Analysis  Post-Mitigation Analysis and Effectiveness  Program level risk adjusted cost and schedule analysis  Supplier selection  Contract Award  Analyze potential risk impact and mitigation posture relating to supplier proposal’s Assess program requirements, supplier capabilities and evaluation criteria to establish RFP Provide insight into where supplier risks affect the program and uncover their true impacts. Compare supplier risk profile at proposal evaluation completion to determine outstanding risk exposure.
  • 9. 8 Omnia Paratus Corporation Acquisition Risk Analysis follows a typical Risk Management process and can be tailored to program specific needs Risk Identificat ion Sets risk parameters for effective risk analysis Examines risks to determine impacts by and across alternatives Identifies risks that may impact cost, schedule, or performance Develops measures for keeping risk at acceptable levels Uses results of risk analysis in assessing alternatives Risk Planning • Collecting SME input, best practices, metrics, etc., to identify possible risks • Documenting risk data into a central repository 1 Risk Identification • Creating standard risk terms and definitions • Developing processes, criteria, and scoring approach for risk Risk Analysis • Conduct probabilistic assessments of risk impacts Risk Mitigation • Identifying, evaluating, and selecting strategies to reduce risk • Developing an actionable risk mitigation plan, with sound rationale (if applicable) Applying Risk Results • Using cumulative risk scores as a vehicle for ranking alternatives 2 3 4 5
  • 10. 9 Omnia Paratus Corporation Building qualitative definitions for risk ratings enables an objective, not subjective quantification of proposal risks  The first step in Risk Planning involves defining standard risk terminology using many different resources, but should be robust and accurate enough to reflect the program’s overall risk tolerance  Developing a robust, well-defined risk analysis process will produce results that reflect program needs; effectively communicates these needs across stakeholders; reduce personal bias in determining relative risk importance; and uncover potential impacts of great importance to the program  Risk criteria is not normative, and are based on available resources, time constraints, and amount/type of information solicited in generating the criteria Performance Metrics The analyst needs to ensure that the units of measure for risk impact reflect program needs. Critical program drivers might include life-cycle costs or metrics reliability (e.g., Mean Time Between Failure). Order of Magnitude If impact thresholds are expressed in percentages (for example) the analyst needs to review those figures within the context of the overall anticipated program cost—e.g., 10% of $5 million program is substantially different than 10% of a $5 billion program. Level of Impact The analyst needs to ensure that there is an adequate number of thresholds for evaluating the risk impact. The more levels of consequence that are utilized, the greater the insight into the need for increased data fidelity/quantity of data the analyst would need to consider. Qualitative vs. Quantitative Qualitative data is represented by probability values ranging from “Very Unlikely” to “Very Likely” , using stakeholder input to assess and weigh the importance of benefits, cost, and risk in relationship to the total analysis or evaluation of benefit, risk, and cost factors that cannot be quantified. Quantitative data is represented by consequence values using linear numerical probabilities (e.g., 0.1, 0.3, 0.5, 0.7, and 0.9), nonlinear numerical probabilities (e.g., 0.05, 0.1, 0.2, 0.4, 0.8), cost estimates, metrics, ranges, or percentages. Considerations for Defining Risk Criteria
  • 11. 10 Omnia Paratus Corporation A Supplier Management Maturity module is used to determine if supplier capabilities comply with program requirements  Framework used to assess each IPTs risk management maturity in terms of People, Process, Technology, and Governance  The optimal goal of the risk enhancement effort is to select supplier cable of bidding based on their “Best in Class” maturity Regular training conducted to enhance skills and capabilities  Dedicated organizational resources All staff is informed and capable of applying mid- to advanced concepts  Applied use of specific processes/tools  Dedicated team resources  In-house core experts, formally trained in basic skills  Dedicated resources do not exist  Limited to individuals who may have had little or no formal training  Minimal understanding or experience in applying basic concepts and principles  Minimal understanding of principles or language  Integral to informed decision-making by upper management  Active use encouraged and rewarded  Part of the organizational philosophy to achieving program success  Top-down commitment by leadership • Accepted as a program management function  Benefits recognized and expected  Upper management requires tracking and reporting  Focus on mitigation effectiveness versus reporting and status tracking  Upper management encourages, but does not require use  Application varied through out program • Process may be viewed as additional overhead with variable benefits • Minimal awareness  Minimal upper management involvement  Tendency to continue with existing processes even in the face of potential failure  Dedicated resources do not exist  State-of-the-art tools and methodologies  Distributed data environment that provides access to all program resources  Standardized and automated reporting capabilities • Integrated set of tools and methodologies  Centralized data environment managed by dedicated team resources • Customizable solutions tailored to program • Few repeatable technological solutions in place • No structured application  Management and tracking tools not in use  Analysis not performed Qualitative/quantitative analysis methodologies employed with emphasis on valid and reliable data sources  Metrics used and reported, with consistent feedback for improvement  External stakeholders actively participate in process Integration into organizational processes and decision-making  Formal processes integrated into different areas of the program  Active allocation and management of budgets  Metrics collected  Key internal stakeholders actively participate in process  Common processes defined and formally documented  Process effectiveness limited to a dedicated team  Qualitative analysis based on ill-defined rating system  Established decision-body forum  Inconsistent application of concepts or principles  Formal risk decision-making body does not exist  Risk reporting and/or metrics is minimal or does not exist  Formal, documented process does not exist 4 – Best in Class3 – High Performance2 – Functional1 – Minimal Capability PeopleProcessTechnologyGovernance NOTIONAL People Process Tech Gov’t Metrics Quality Capacity Average Rating 3 4 4 1 2 2 1 2.42 2 3 - 3 3 1 1 1.85Supplier B 3 3 4 2 3 3 2 2.85Supplier C 3 3 4 4 3 3 2 3.14Supplier D Supplier A  Supplier’s will be scored based on the maturity model to establish a list of qualified suppliers for proposal solicitation NOTIONAL
  • 12. 11 Omnia Paratus Corporation Once proposals have been received a technical evaluation should be conducted for clarifications prior to identifying supplier risks Proposal Clarification Items Topic General Area Staffing Questions  Is the team available as indicated in the Proposal?  Additionally, a further elaboration on local staff/ qualifications is required Ref. P. 60 Approach  A further elaboration on the timeline is required, especially on the 9 days timeframe proposed for defining the architecture and standards  A show-case of the deliverables must be provided  How realistic are the assumptions on available data and Organization staff? P. 46 Assumptions  What is the impact on costs and timeline of a provision of a documentation in other languages? P. 18 Phase 0: Kick-Off Phase 1: Baseline Phase 2: Best Practice Review Alignment Tools Approach  How will Bidder ensure the alignment of the PM tools to the Organization’s PMO?  How will the automatic “systems” feed work (XML). Is it a requirement?  How will any incompatibilities of import functions affect the suggested timeline?  What if the Organization wants to use other EA tools, like ARIS?  Further elaboration on “GEAS” layers required. A mapping to the Organization framework layers is needed as well  Explanation of the rationale for selecting Singapore, Australia and Canada  Is the access to those countries only via databases/benchmarks or “real” contact? P. 34 P. 36- 38 P. 43 Phase 3: Definition of Architecture & Standards Imperatives  No reference given, on how the Organization’s imperatives will be ensure – therefore, a further elaboration on the approach is required P. 45 Answer Given Issue Resolved  All questions to a Bidder should be compiled and sent in a formal email / fax  All Bidders should be given ample and the same amount of time to respond to clarifications requested
  • 13. 12 Omnia Paratus Corporation The next step to the risk assessment process is to identify risks that may impact a program’s cost, schedule, or technical performance…… Risk Identificat ion  The risk analysis focuses on risks that affect each supplier’s ability to ensure program success (i.e., ongoing, uninterrupted support to the operational forces).  Although identification of risk relies on the skill, experience, and insight of subject matter experts and risk personnel, the methods and tools for initiating the identification of risk may vary  As such utilizing risk categories such as these bolsters risk identification… “Common” Risks Scope of Work Testing & Integration Material Availability Suppler Metrics Schedule Budget Constraints Qualification Requirements Supplier Capabilities Performance Metrics Resources Lessons Learned ManagementComponents Program Requirements Sensitivity Analysis
  • 14. 13 Omnia Paratus Corporation Technical …each risk is then reviewed against a set of Program-specific impact definitions, with the highest rating used to determine overall risk severity Cost ( % over of cost target) Supplier Delivery Schedule Existing technology does not exists Existing technology exists, but has not been proven Supplier has never built component before, Flight Test required Supplier has never built component before, 1st article required Supplier has never built component before Supplier built similar component, Flight Test required Supplier has built a component similar in nature, 1st article required Supplier has built identical component for other Programs Supplier has previously built component Incumbent supplier Level Disastrous Severe Critical Substantial Significant High Moderate Medium Low Minimal Technical = High 3 Schedule = Critical 1 Cost = High 2 Consequence Rating = Critical > 6.00% 5.01 – 6.00% 4.01 – 5.00% 3.01 – 4.00% 2.01 – 3.00% 1.01 – 2.00% .76 – 1.00% .51 - .75% .26 - .50% < .25% >5 month slip in MRP requirements 4-5 month slip in MRP requirements 3-4 month slip in MRP requirements 2-3 month slip in MRP requirements 1-2 month slip in MRP requirements < 1 month slip in MRP requirements Risk erodes 100% of schedule margin Risk erodes 51 – 75% of schedule margin Risk erodes 26 – 50% of schedule margin Risk erodes < 25% of schedule margin
  • 15. 14 Omnia Paratus Corporation Risk # WBS Impact IMS Task ID Risk Description Risk Rating Cost Impact ($K's) Schedule Impact (Days) 1 53 Lack of supplier staff to support delivery requirements 3 8 31.00 $7,500 88 2 71 CCB is a development item 5 9 71.25 $8,500 118 3 101 Supplier X's On Time Delivery rating subpar 2 10 28.50 $9,500 150 4 88,97 Requirements of flux capacitor increase the need for Supplier X System Test & Eval staff and resources 4 6 32.25 $5,500 52 5 70 Lack of technical capability in power generation / storage hardware may cost / schedule over runs 1 3 3.30 $2,500 18 6 67,97 Tooling re-use approach may not be compatible with new technology 4 10 71.25 $9,500 150 Prob. Conseq. Risks are then integrated into the suppliers proposal to determine impacts of risks based on the risk ratings identified  The risk database transposes risk rating into cost and schedule impacts in quantifiable dollar and days  WBS and IMS task are later used to integrate supplier risk adjusted proposal into the Program’s Budget and IMS
  • 16. 15 Omnia Paratus Corporation Monte Carlo Analysis is used in calculating risks associated with supplier cost impacts to program budget….  This chart example utilized Crystal Ball to perform Monte Carlo, multiple types of software exists and can be used for this analysis  Pick a Confidence Level based on program maturity and requirements  This example highlights the 75th Percentile – 75% of costs are below the line, 25% of costs are above – The 75% CL is $49,448
  • 17. 16 Omnia Paratus Corporation ….as well as supplier risks that have the potential for disrupting a program’s Operations schedule.  The analysis example utilized SCRAM to run the schedule risks analysis  Confidence Level (CL) picked for cost is also used for schedule to determine how risk may impact a supplier’s delivery schedule
  • 18. 17 Omnia Paratus Corporation Schedule & Cost Risk Assessment Module (SCRAM) is a MS Project add-in and FREE for use on NASA projects.  SCRAM’s capabilities compare with that of Pertmaster, @Risk and Risk++  Extremely user friendly and reliable  Customizable aspects not available with other tools  Compatible will all MS Office Products
  • 19. 18 Omnia Paratus Corporation Once Monte Carlo simulations are complete and confidence levels selected, supplier bids can be compared based on potential risk impacts Proposed $ 494,958 $ 465,135 $ 514,453 10% $ 507,645 $ 512,622 $ 547,565 20% $ 509,542 $ 523,002 $ 555,345 30% $ 510,834 $ 530,892 $ 561,676 40% $ 512,016 $ 537,855 $ 567,689 50% $ 513,090 $ 544,29 $ 573,385 60% $ 514,241 $ 551,640 $ 579,471 70% $ 517,449 $ 552,673 $ 585,912 80% $ 520,923 $ 565,114 $ 594,321 90% $ 523,007 $ 582,310 $ 607,304 100% $ 629,484 $ 589,685 $ 711,107 Supplier A Supplier B Supplier C $700,000,000$600,000,000$500,000,000 Confidence Level Chosen Supplier B Supplier A Supplier C
  • 20. 19 Omnia Paratus Corporation $497,042 $482,331 $427,436 $0 $50,000 $100,000 $150,000 $200,000 $250,000 $300,000 $350,000 $400,000 $450,000 $500,000 $550,000 $600,000 $650,000 Supplier A Risk Adjusted Bid Supplier B Risk Adjusted Bid Supplier C Risk Adjusted Bid Side-by-side risk exposure calculations provides leadership with comparative insights into supplier potential costs impacts ProgramCost(inthousands) $39,100 Supplier Initial Cost Risk Exposure All costs reported at the 90% confidence interval. Analysis will lead to a cumulative assessment of the total risk exposure and the potential impact to program budget. Component budget $500K Potential Risk Impact $25K Potential Risk Impact $99K Potential Risk Impact $179K
  • 21. 20 Omnia Paratus Corporation 0 1 2 3 4 5 6 7 8 6/2/10 6/6/10 6/10/10 6/14/10 6/18/10 6/22/10 Material Resource Planning requirement for this component is 04/25/2010 Risk adjusted delivery schedules are then compared to determine the potential risk impact to program’s operations Supplier’s Initial Proposal Supplier A B C Proposed Schedule 04/18/2010 04/04/2010 04/21/2010 Risk Adjusted Schedule 05/13/2010 07/18/2012 08/21/2012 0 1 2 3 4 5 6 7 8 9 5/3/10 5/7/10 5/11/10 5/15/10 0 0.5 1 .5 2 .5 3 .5 4 4.5 7/7/10 7/11/10 7/15/10 7/19/10 7/23/10 7/27/10 7/31/10 8/4/10 8/8/10 Illustrative IllustrativeIllustrative Supplier A Supplier B Supplier C
  • 22. 21 Omnia Paratus Corporation The risk-adjusted cost and schedule results are then compared for awarding contracts on a risk averse path  Initial review of Supplier bids would indicate “C” as the supplier of choice  Based on Supplier B & C delivery metrics and potential risk, schedule impact could result in more than a 3 months past MRP requirements  Based on supplier C’s lack of technical capabilities and schedule risk to operations, risk impact could equate to ~$100K over component budget  Based on risk adjusted Cost & Schedule proposal analysis supplier “A” should receive program consensus for contract award based on least amount of risk exposure to the program Supplier’s Risk Adjusted Proposal Supplier A B C Risk Adjusted Cost $ 523,007 $ 582,310 $ 607,304 Risk Adjusted Schedule 05/13/2010 07/18/2012 08/21/2012 Supplier’s Initial Proposal Supplier A B C Proposed Cost $ 497,042 $ 482,331 $ 427,436 Proposed Schedule 04/18/2010 04/04/2010 04/21/2010
  • 23. 22 Omnia Paratus Corporation Given these potential benefits, a few key considerations are worth noting  In order to have a successful portfolio risk management process, it’s important that the constituent components of the program have sufficiently mature supply chain management and risk management processes.  Integration of acquisition risk analysis into a program’s budget and schedule is necessary to capture the magnitude of potential program risk impact by a single supplier  Identifying risks within a proposal enables forward looking program management that can be streamlined into existing risk database’s for future risk management planning and mitigation.  Qualitative risk analysis provides enhanced proposal evidentiary support and solid justification for awarding contracts  The success of a supply chain risk management program requires the consistent and active support of program leadership in order to be successful.
  • 24. 23 Omnia Paratus Corporation For more information on how acquisition risk analysis can be applied to your specific challenges, please:  Contact: James Taylor Huntsville, Alabama 310-462-6878