Your SlideShare is downloading. ×
  • Like
Business Case Guidelines
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Business Case Guidelines

  • 2,108 views
Published

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,108
On SlideShare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
82
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Federal Aviation Administration FINAL January 6, 2010 Business Case Analysis Guidelines Approved By: ________________________________________Date:_____________ Kristen Burnham, Director, Investment Planning and Analysis (AJF-3)
  • 2. 1.0 Introduction...................................................................................................................................... 1.1 Why We Need a Business Case......................................................................................... 1.2 Business Case Context....................................................................................................... 2.0 Business Case Analysis......................................................................................................... 2.1 Cost Analysis..................................................................................................................... 2.1.1 Risk-Adjusted Life Cycle Cost Estimate................................................................... 2.1.1.1 Work Breakdown Structure................................................................................... 2.1.1.2 Cost Estimating Methodologies............................................................................. 2.1.1.2.1 Analogy..................................................................................................................... 2.1.1.2.2 Parametric Estimating and Cost Factors.................................................................. 2.1.1.2.3 Engineering/Bottom-Up Estimating........................................................................ 2.1.2 Market Data............................................................................................................... 2.1.3 Cost Basis of Estimate Documentation..................................................................... 2.2 Benefit Analysis................................................................................................................. 2.2.1 Benefits Methodology............................................................................................... 2.2.1.1 Analogy.................................................................................................................. 2.2.1.2 Parametric Analysis............................................................................................... 2.2.1.3 Extrapolation (Engineering Method)..................................................................... 2.2.1.4 Subject-Matter Expert Opinion.............................................................................. 2.2.2 Initial Benefits Estimate............................................................................................ 2.2.3 Final Benefits Estimate.............................................................................................. 2.2.4 Benefits Basis of Estimate Documentation............................................................... 2.3 Business Case Risk Analysis............................................................................................. 2.4 Schedule Analysis.............................................................................................................. 2.5 Economic Analysis............................................................................................................ 2.5.1 Net Present Value...................................................................................................... 2.5.2 Payback Period.......................................................................................................... 2.6 Additional Assessments..................................................................................................... 2.6.1 Architecture Impact Assessment............................................................................... 2.6.2 Human Factors Assessment....................................................................................... 2.6.3 Information Security Assessment.............................................................................. 2.6.4 Safety Assessment..................................................................................................... 3.0 quality assurance.................................................................................................................... APPENDICES APPENDIX A: BUSINESS CASE REQUIREMENTS...........................................................A- APPENDIX B: BUSINESS CASE TEMPLATE.....................................................................B- APPENDIX C: HELPFUL WEBSITES...................................................................................C- APPENDIX D: LIST OF ACRONYMS...................................................................................D- 2
  • 3. 1.0INTRODUCTION The Office of Investment Planning & Analysis (IP&A) sponsors these guidelines which describe the activities undertaken during Business Case Analysis (BCA) required for the Initial Investment Decision (IID) and Final Investment Decision (FID). IP&A is responsible for ensuring that new, proposed, and existing NAS investments meet established business case and economic criteria by completing schedule and risk assessments; validating the business justification of proposed capital investments; and ensuring business case and investment analysis policies, procedures, standards and training are established, followed, and maintained. AFC-300 performs this function for ITEB capital investments. The focus is on improving investment decision-making and ensuring the agency is postured for change. 1.1Why We Need a Business Case The reason for developing a Business Case is to justify the resources and capital investment necessary to bring change to fruition. Business case preparation is also intended to ensure that acquisition preparations are complete to ensure FAA receives maximum value for the resources expended. However the Business Case is not simply a financial document. The Business Case is the one place where all relevant facts are documented and linked together into a cohesive story. This story tells people about the why, what, when, and how: • Why is the project needed (issues & opportunities)? • What is the recommended solution(s)? • What are the risks, advantages, and disadvantages of the existing and proposed alternatives? • What will happen if the effort is not undertaken (the do nothing scenario)? • When will the solution be deployed? • How does the solution address the issues or opportunities (benefits)? • How much money, people, and time will be needed to deliver the solution and realize the benefits? The Investment Decision Authority will make decisions for the agency on whether to proceed with the proposed investment or not, based in large part, on the answers to these questions. The business case forms the framework for those decisions. If the decision is made to proceed with the investment, the program needs to operate within the context of the approved business case and investment analysis. If, as the program proceeds, conditions change which take the program beyond that context, the program will need to be revisited and the investment analysis reviewed with the changed conditions to determine if the program should be continued, and with what potential modifications. Business Case Analysis Guidelines January 6, 2010 3
  • 4. 1.2Business Case Context Investment analysis (IA) is the detailed analysis of alternative ways to meet a mission capability shortfall or to take advantage of a technological opportunity identified during Concept and Requirements Definition (CRD) (see Investment Analysis Process on the AMS FAST web site). It is conducted as a partnership between the sponsoring and operating organizations to ensure the critical needs of the users are satisfied by a solution that is affordable. Business case analysis (BCA) is the economic part of investment analysis and identifies the risk versus reward for alternative courses of action. Figure 1 shows the relationship of CRD, IA, BCA and Contractual Products. 0100090000030202000002008a01000000008a01000026060f000a03574d464301000000000001 0041330000000001000000e802000000000000e8020000010000006c00000000000000b0070000 2c000000b007000000000000000000007d4000008d20000020454d4600000100e80200000e0000 0002000000000000000000000000000000c0120000c0180000cb0000000c010000000000000000 000000000000f8180300e0160400160000000c000000180000000a000000100000000000000000 00000009000000100000003c0f0000af070000250000000c0000000e000080120000000c0000000 10000005200000070010000010000009cffffff00000000000000000000000090010000000000000 4400012540069006d006500730020004e0065007700200052006f006d0061006e0000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000000000000000000e7360930000000000400 00000000ae30243709300000000047169001000002020603050405020304877a00200000008008 00000000000000ff01000000000000540069006d00650073002000000065007700200052006f00 6d0061006e000000c23f00003006000000000000e80d56012982cc300000000018fc100032b5023 018fc10004c5eaf3030fc10006476000800000000250000000c00000001000000180000000c0000 0000000002540000005400000000000000b00700002c000000b00700000100000055558740815a 8740000000000a080000010000004c0000000400000000000000000000003c0f0000b107000050 00000020001b002d00000046000000280000001c0000004744494302000000ffffffffffffffff3d0f0 000b0070000000000004600000014000000080000004744494303000000250000000c0000000e0 000800e000000140000000000000010000000140000000400000003010800050000000b0200000 000050000000c020b011002040000002e0118001c000000fb020200010000000000bc020000000 00102022253797374656d00000000000000000000000000000000000000000000000000000400 00002d01000004000000020101001c000000fb02f2ff00000000000090010000000004400012546 96d6573204e657720526f6d616e0000000000000000000000000000000000040000002d0101000 50000000902000000020d000000320a17010000010004000000000010020b01205206000400000 02d010000030000000000 Figure 1. Business Case Context 2.0BUSINESS CASE ANALYSIS Business case analysis captures the reasoning for initiating a project or task. It is tailored by Acquisition Category (ACAT) level and investment decision point. The more complex and demanding analysis is required for the more complex and demanding investments (ACAT 1 vice ACAT 5), and the more complex and demanding decisions (FID vice IID). There are five
  • 5. elements to a BCA: Cost, Benefit, Risk, Schedule and Economic Analyses. For the IID, the initial business case analysis considers at least three alternative approaches to achieving the desired capability. For the FID, the final business case analysis requires a thorough and detailed analysis of the alternative selected at IID. In each case, alternatives are evaluated in terms of technical, operational, cost, and benefit trade-offs against the Legacy Case or Status Quo. Business Case Analysis Guidelines January 6, 2010 5
  • 6. Table A-1 in Appendix A summarizes business case analysis requirements by ACAT and investment decision point. Some major differences between requirements are: For IID, ACATs 1/2 require Monte Carlo simulation for the risk analysis while ACAT 3 does not; ACATs 1/2 require probabilistic economic analysis with results prepared at the 80% confidence level, while ACAT 3 only requires a deterministic analysis. ACAT 4/5 (New Functionality) does not require schedule analysis and ACAT 4/5 (Tech Refresh) does not require an initial business case analysis. For FID, ACATs 1/2 require detailed resource-loaded schedule analysis while a "top level" schedule analysis is sufficient for ACAT 3; ACATs 1/2 require probabilistic economic analysis with 80% confidence level, while ACAT 3 still requires a probabilistic analysis but without a specific level of confidence specified. ACAT 4/5 (New Functionality) does not require Monte Carlo simulation or probabilistic analysis and ACAT 4/5 (Tech Refresh) does not require Net Present Value (NPV) or Payback Period analyses. 2.1Cost Analysis There are four components to cost analysis: (a) full risk-adjusted life cycle cost estimate (LCCE); (b) level of Work Breakdown Structure (WBS) to which the full risk-adjusted LCCE is conducted; (c) methodologies used to develop costs; and (d) use of data provided by the market and/or offerors. 2.1.1Risk-Adjusted Life Cycle Cost Estimate A full risk-adjusted LCCE is needed for all ACAT categories for both the initial and final investment decisions. What varies is whether life cycle costs are captured for all alternatives (IID) or for just the preferred alternative (FID). What varies by ACAT level and decision point are the other three components, i.e., WBS level, cost estimating methodologies, and use of data from the market and/or offerors. (See Table A-1, Business Case Requirements). The Investment Analysis Team evaluates the alternative means of achieving the required capability by WBS in terms of cost, schedule and technical risk. Where risk is identified, mitigation efforts are developed, costed and added to the appropriate WBS by fiscal year. For ACAT 1 & 2 unknown and unmitigated risks, Monte Carlo simulation (a statistical simulation technique) analysis is used and the costs applied to the appropriate WBS by fiscal year. For ACAT Level 3, Monte Carlo simulation is not required, and Risk is applied at the WBS level. Risk adjustments for ACAT Levels 4 and 5 can be applied at either the WBS level or added to the total estimate (added to the “bottom line”); Monte Carlo simulation is not required. The initial business case analysis is conducted during the Initial Investment Analysis phase and it considers at least three alternatives. This means that three risk adjusted LCCEs must be developed. Each of the alternatives must be measured against the Legacy Case or Status Quo. The “Legacy Case” is the existing and planned near-term capability including the operating and planned near-term set of assets, systems, facilities, people and processes that perform an existing
  • 7. FAA function. In producing a Legacy Case estimate, it is often necessary to add new equipment/ systems to maintain current functionality. Including costs of equipment to maintain current capability is acceptable so long as no new functionality is added. The Legacy Case must be evaluated in the same manner as an alternative (cost, performance, benefits, risks, and schedule) and assumes that no new functionality will be added to meet the new requirements. The final business case analysis refines and updates the costs of the alternative selected at IID and the Legacy Case. A contract specific Independent Government Cost Estimate (IGCE) is prepared for the preferred alternative. The IGCE is an estimate of the program acquisition costs supported by factual or reasoned data and documentation, describing how much FAA could reasonably expect to pay for needed supplies or services. FAA internal costs (i.e., those program costs that will not be purchased from the vendor, such as IOT&E, organic maintenance, etc.) are excluded. The IGCE is used by the Acquisitions office to evaluate vendor responses to FAA proposals. 2.1.1.1 Work Breakdown Structure The FAA standard life cycle WBS shall be used when developing cost and schedule estimates. A WBS is the complete set of activities that may be accomplished to provide a solution that satisfies an FAA mission need. Solutions include products and services such as hardware, software, facilities, communications services, government employees, technical assistance services, infrastructure, training, procedures, etc. The elements in the WBS represent activities, not the resources needed to accomplish the activities. After the activities are defined, resources are identified, time phased, and costs are estimated. For the initial business case analysis, estimates should be prepared at a level 2 WBS as a minimum. Final business case analysis estimates should be prepared at a level 3 WBS as a minimum. A level 2 WBS defines the major program segments. For Systems Engineering, a level 2 WBS includes all technical and management activities associated with a specific solution that concentrates on the definition, design, and application of the whole product throughout the program life cycle. These activities include planning, directing, and controlling a totally integrated engineering effort of a solution. Systems engineering consists of such functional disciplines as requirements definition and allocation; analysis, design, and integration; value engineering; supportability, maintainability, and reliability engineering; quality assurance; interface management; configuration management; human factors; security; safety engineering; and specialty engineering. The costs for these sub-areas are “rolled-up” into WBS 3.2, Systems Engineering. A level 2 WBS example is: 3.0 Solution Development = Level 1 3.1 Program management = Level 2 3.2 Systems engineering = Level 2 3.3 Hardware/Software design, development, procurement, production = Level 2 Business Case Analysis Guidelines January 6, 2010 7
  • 8. A level 3 WBS includes all activities associated with planning, managing, supporting, executing and maintaining system engineering process and work. A level 3 WBS example is: 3.0 Solution Development = Level 1 3.1 Program management = Level 2 3.2 Systems engineering = Level 2 3.2.1 Systems engineering management = Level 3 3.3 Hardware/Software design, development, procurement, production 2.1.1.2 Cost Estimating Methodologies Many techniques can be used for cost estimating, from simple arithmetical calculations to complex mathematical models with numerous variables. Some of the techniques are: • Analogy: Used early in the acquisition management life cycle based on a one-to-one comparison with an existing product similar to the product you are designing; • Parametric: Uses statistical analysis from a number of similar products and their relationship to your product. • Engineering: A bottom-up estimate using the detailed WBS structure to price out discrete components, such as material, design hours, labor, etc. • Extrapolation-from-actual-costs: Method used late in the acquisition life cycle after actual cost data are available from the same system at an earlier time. The more complex and demanding methodologies are used for the more complex and demanding decisions (FID vice IID), and the more complex and demanding investments (ACAT 1 vice ACAT 5). The first three of the above methodologies are appropriate for business case analysis. 2.1.1.2.1 Analogy The analogous or comparative method takes into consideration that no new program, no matter how advanced, represents a totally new product. Most new programs originate or evolve from existing programs or simply represent a new combination of existing components. This method of estimating uses this idea as a foundation for estimating new components, subsystems, or total systems. Simply stated, it uses actual costs of a similar existing or past program, and adjusts for complexity, technical, or physical differences to derive the new product estimate. This method is normally chosen when there is insufficient actual cost data to use as a basis for a detailed approach but an analogous item exists on which to base an estimate. A detailed engineering assessment is required to ensure the best analogy has been selected and proper adjustments are made. The ability to break the estimate down into a low-level of detail further enhances the credibility of the estimate, since separate analogies can be chosen for each
  • 9. component. There are two limitations in using an analogous approach. First, is the requirement for a detailed program and technical definition of both the analogous product as well as the product being estimated. Engineering judgment becomes the mainstay of this approach and, at the same time, a limitation. Without access to sound engineering support, this methodology is difficult to employ. Secondly, once the technical assessment has identified the analogous product, actual cost data on that product must be acquired. Without this, the transition from the analogous product to the current product cannot be made. 2.1.1.2.2 Parametric Estimating and Cost Factors Parametric cost estimating is the use of historical cost data and statistical techniques to predict future costs. The implicit assumption of parametric cost estimating is that the same forces that affected cost in the past will affect cost in the future. The major advantage of using a parametric methodology is that the estimate can usually be conducted quickly and is easily replicated. Parametric estimating is a technique that employs one or more Cost Estimating Relationships (CERs) and associated mathematical relationships and logic. Statistical techniques are used to develop CERs that tie the cost or price of an item (e.g., a product, good, service, or activity) to one or more independent variables, that is, cost drivers. Parametric models are used in the early design stages to get an idea of how much the product (or project) will cost based on a few physical attributes (such as weight, volume, and power). The output of the parametric models (an estimated cost) is used to gauge the impact of design decisions on the total cost. Awareness of the effect engineering design decisions have on total cost is essential to developing a product that is both economically and technically sound. Parametric Estimating and Cost Factors is acceptable for all ACAT Levels at IID and for ACAT Levels 4 and 5 at FID. Use of the Pocket Estimating Guide (PEG), internally developed factors, or the use of other government agency estimating guides (i.e., NASA) is acceptable. An explanation of why a particular factor was selected should be included. PEGs were developed using recent FAA data. They provide WBS cost estimating factors and permit the development of rough order of magnitude (ROM) estimates when only Prime Mission Equipment (PME) data is available. PEGs exist for both development and production efforts. 2.1.1.2.3 Engineering/Bottom-Up Estimating Bottom-up estimating methodology rolls up individual estimates for each element into the overall estimate. This costing methodology involves the computation of the cost of a WBS element by estimating at the lowest level of detail (often referred to as the “work package” level) wherein the resources to accomplish the work effort are readily distinguishable and discernable. Often the labor requirements are estimated separately from material requirements. Overhead factors for cost elements such as Other Direct Costs (ODCs), General and Administrative (G&A) expenses, materials burden, and fee are generally applied to the labor and materials costs to complete the estimate. A technical person who is very experienced in the activity typically Business Case Analysis Guidelines January 6, 2010 9
  • 10. works with the cost analyst, who prepares these engineering build-up estimates. Cost estimates for ACAT Levels 1, 2 and 3 at FID should be prepared using the engineering build up or “bottom-up” technique. Also, the cost estimate for the prime contractor should be based on the data from the final SIR. 2.1.2Market Data The marketplace is an excellent source of information and data for business case analyses. The extent and depth of market research and analysis is tailored to the individual requirement, estimated dollar value, complexity, urgency, and past experience. Market research may range from a telephone call or review of purchase histories to formal market surveys or solicitations requesting information. Market research means collecting and analyzing information about vendor capabilities to satisfy FAA’s requirements. This research can help discover novel or innovative solutions, eliminate excessively complex or unnecessary requirements, identify non-value added costs, and improve vendor’s responsiveness to subsequent solicitations. 2.1.3Cost Basis of Estimate Documentation The Cost Basis of Estimate (BOE) provides a record of the procedures, ground rules and assumptions, data, environment, and events that underlie a cost estimate’s development or update. Good documentation supports the cost estimate’s credibility, aids in the analysis of changes in program cost, enables reviewers to effectively assess the cost estimate, and contributes to the population of FAA databases for estimating the cost of future programs. For IID, no specific documentation format is required; however, cost data should be sufficiently complete and well organized such that a cost estimating professional can use the data and methodology by themselves to understand and assess the estimate. For FID, ACAT Levels 1, 2, & 3, full Cost BOE documentation is required for each WBS element. It should describe the derivation of the estimated cost in sufficient detail to allow an independent reviewer to determine whether the estimate is comprehensive, accurate, and realistic. Where possible, estimates should be documented within the EXCEL spreadsheets; otherwise, a separate Word document is necessary. 2.2Benefit Analysis Benefits analysis is the process of determining benefits resulting from the implementation of FAA projects and programs. Benefits may be both quantifiable and non-quantifiable in nature. They may result from operational efficiencies, cost avoidance, improved safety, or improved security among others. These savings may accrue to the users of the NAS, the FAA, or the public. A variety of techniques are used including analogy, parametric analysis, extrapolation and
  • 11. subject-matter expert opinion. In defining benefits for a program, a wide perspective of stakeholders needs to be considered including the FAA, other government agencies (e.g., DOD, DHS), user communities (e.g., airlines, airports, the flying public) as well as the general public. In quantifying benefits, the analyst compares the “status quo” or legacy environment to the projected future environment once the technology, process, infrastructure, facility, or system has been implemented. Because the operational environment in which the technology/process/system/facility is to be employed may impose severe restraints on its use, it is important that the operational environment be considered in determining the true level of estimated benefits. Quantitative benefits identified in the benefits analysis must be able to be tracked so that the program will be able to report its progress in achieving the benefits claimed. 2.2.1Benefits Methodology 2.2.1.1 Analogy The analogous or comparative method takes into consideration that no new program, no matter how advanced, represents a totally new product. Most new programs originate or evolve from already existing programs or simply represent a new combination of existing components. This method of estimating uses this idea as a foundation for estimating new components, subsystems, or total systems. Simply stated, it uses actual benefits of a similar existing or past program, and adjusts for complexity, technical, or physical differences to derive the new product estimate. Such estimates generally are characterized by use of a single historical data point serving as the basis for an estimate or portion thereof. Use of analogy estimating methods is advisable when the new product is primarily a combination of existing subsystems, equipment, or components for which recent and complete historical benefit data are available. Analogy methods are most useful in situations where rapidly advancing technology and acquisition strategies cause a parametric model database to become antiquated quickly. 2.2.1.2 Parametric Analysis The parametric method estimates benefits based upon various characteristics or attributes of the product being estimated. It depends upon the existence of a causal relationship between product costs and these parameters. Parametric estimating is the process of using mathematical equations that relate to one or more physical or performance characteristics of the item being estimated. Parametric estimates are often used in the early phases of a product’s life cycle. At that stage of the life cycle, basic physical or performance characteristics may be available, but detailed designs may not be. Thus, parametric approaches may be the only option. Business Case Analysis Guidelines January 6, 2010 11
  • 12. 2.2.1.3 Extrapolation (Engineering Method) The engineering method (also referred to as detailed, grassroots, or bottoms-up estimating) is an estimate that starts at a very low level of detail and builds up to a total benefits estimate. This type of estimate is used when detailed data are available on a product. 2.2.1.4 Subject-Matter Expert Opinion Subject matter expert estimating is a judgmental estimate performed by an expert in the area to be estimated. The Delphi Technique may be employed when surveying a number of experts independently to reach a consensus. This methodology is limited by the availability of expert judgment and the credibility of that judgment. This approach is best used as a cross check against an existing estimate or in combination with other methodologies. 2.2.2Initial Benefits Estimate The metrics, methodology, and techniques used for initial business case benefits analysis should support the shortened timeframes generally found with IID decisions. Initial business case benefits analysis should be consistent with the Concept Requirements Document (CRD) Shortfall Analysis which provides the basis for quantifying program benefits. A benefits analysis is conducted for each alternative taking into account those benefits that may “come on line” or be realized differently for each alternative. Benefits estimates for an IID are generally more aggregated than those for an FID and are reflective of general mission performance goals. Even so, ACAT Levels 1, 2, 3, and 4/5 (new functionality) all require a full risk-adjusted life cycle benefits analysis similar in scope and depth to its corresponding cost estimate. Due to the higher investment required for ACAT Level 1 and 2 programs, the benefits analysis for these programs should be prepared more rigorously and with more detail than ACAT Levels 4 and 5. 2.2.3Final Benefits Estimate In the final business case analysis, ACAT Levels 1, 2, 3, and 4/5 (new technology) require a full risk-adjusted life cycle benefits analysis for the selected alternative. This is similar in scope and depth to its corresponding cost estimate. ACAT level 4/5, technology refresh, requires a cost avoidance analysis (example: cost of replacement versus cost of repair) and denial of operational capability/service assessment. In conducting the final business case analysis, a full risk-adjusted benefits analysis should be done for every type of quantifiable benefit claimed in the initial business case analysis. For example, if benefits are claimed for both aircraft operational efficiency as well as personnel savings, a full risk-adjusted benefits analysis needs to be presented on both benefit types. If more than one benefit driver exists for a particular benefit type, a full risk-adjusted benefits analysis should be presented for each. For example, if operational efficiency benefits are claimed from multiple technology drivers, the independent benefit contribution of each technology needs to be shown. To emphasize, the independent contribution of each benefit driver needs to be determined and documented. Care needs to be exercised that the time-phased benefits stream accurately reflects the benefits arriving from the time-phased investment stream.
  • 13. Once the physical contribution of each quantifiable benefit driver is determined, its economic benefit needs to be determined. Official FAA and Department of Transportation (DOT) economic factors should be used for monetizing these benefits; exceptions need to be documented. Also, any non-quantifiable benefits arising from the investment should be documented. 2.2.4Benefits Basis of Estimate Documentation For IID, no specific Benefits BOE documentation is required; however, benefit data should be sufficiently complete and well-organized such that a reviewer can use the data and methodology by themselves to understand and assess the estimate. In documenting the benefit analysis, for each benefit type claimed, the following needs to be identified: the benefit drivers; the methodologies/techniques/models used for estimation; the data sources used for model parameter calibration and input data, as well as any assumptions used or embedded in the data (this is especially important when third party data has been used); and the level of risk assumed for each benefit type. 2.3Business Case Risk Analysis Business case risk analysis is an objective evaluation of each alternative to determine the probability of an undesirable event occurring/not occurring during implementation and the significance of the consequence of the occurrence. It is a process in which a group of programmatic, technical and analytical specialists review the cost and benefits estimates and supporting ground rules, assumptions and BOE. If the analysis indicates that an undesirable event may arise, the potential impact(s) resulting from such an occurrence/nonoccurrence must be evaluated. As a minimum, the areas of risk to be analyzed for each alternative are: costs, benefits, schedule, and technical. For ACAT Levels 1 and 2, risk is identified and mitigation efforts applied at the FAA WBS level specified in Table A-1 in Appendix A. Monte Carlo simulation is required and complements risk analysis performed by subject matter experts. Monte Carlo accounts for and captures “unknown” or unmitigated programmatic risk. A model, such as @Risk or Crystal Ball is required. Risk adjustments are applied to the WBS by fiscal year. For ACAT Levels 3, 4 and 5 (new functionality) risk is applied at the FAA WBS level specified in Table A-1, but does not require Monte Carlo Simulation modeling. Risk is identified, mitigation techniques are devised, their costs estimated and added to the appropriate WBS element. For ACAT Levels 4 and 5, risk may be added to either the appropriate WBS element or to “the bottom line”. Business Case Analysis Guidelines January 6, 2010 13
  • 14. 2.4Schedule Analysis A program schedule, including assumptions, should be developed for each alternative. The summary should include a discussion of activities taken to generate risk-adjusted schedules for each alternative. For the initial business case analysis, for ACAT Levels 1, 2 and 3, a “top level” schedule analysis is required. It includes assessment of major acquisition and programmatic milestones, and supporting activities, aligned with cost and benefit estimates. This assessment can be as simple as a GANT chart. It does not include critical path analysis, nor is it a resource-loaded schedule using an automated scheduling tool such as Primavera. For ACAT Levels 4 and 5 (new functionality), the schedule in the Implementation Strategy and Planning Document (ISPD) is sufficient. Except for the depth of the schedule analysis, final business case analysis requirements for ACAT Levels 1, 2 and 3 are the same. ACAT Levels 1 and 2 require a detailed resource-loaded schedule analysis, generally requiring the use of an analysis tool such as Microsoft project or Primavera. ACAT Level 3 requires a top-level schedule analysis consisting of major procurement and implementation milestones. ACAT Levels 4 and 5, New Functionality and Tech Refresh, do not require a schedule analysis beyond what is in the Implementation and Planning Document. 2.5Economic Analysis Economic analysis is the process of translating a program’s Cost/Benefit Analysis into NPV and other financial statements about FAA investments. Economic analysis provides a systematic approach to answer these questions – what objectives should be pursued and how they should be accomplished. The FAA generally uses NPV as the standard criterion for deciding whether a program can be justified on economic principles. 2.5.1Net Present Value The NPV indicates an investment’s net value in today’s dollars. All costs and benefits are adjusted to "present value" by using discount factors to account for the time value of money. NPV is a way of making costs and benefits occurring in different years commensurable. It is the algebraic combination of the present value of costs and benefits. OMB Circular A-94 establishes NPV as the standard criterion for deciding whether a government project’s costs can be justified on economic principles. NPV forecasts when the investment will generate sufficient cash flows to repay the invested capital and provide the required rate of return on that capital. Because all cash flows are discounted back to the present time, the NPV compares the difference between the present value of the benefits and costs, and takes into account what the project gives up to get these benefits, or the opportunity costs of both cash flows.
  • 15. ACAT Levels 1 and 2 require a probabilistic analysis (statistical, Monte Carlo) for both IID and FID. For program justification purposes, costs are estimated at the 80% confidence level, i.e., the estimate that has an 80% probability of under-run in actual program execution. For benefits, it is the estimate that has an 80% probability of being overrun in actual program execution. For ACAT Level 3, a Deterministic Analysis is acceptable with full Monte Carlo simulation at the 80% level required at FID. ACAT Levels 4 and 5, new technology, require a deterministic economic analysis, while ACAT Levels 4 and 5, tech refresh, do not require NPV. 2.5.2Payback Period The payback period helps to answer the question “how long will it take to make back the money spent on the investment?” This is beneficial information to decision-makers because out-year benefits are often less certain than benefits that occur early in the life cycle. The payback period represents the time required for the cumulative savings to equal to the cumulative value of investment. The payback period measures the time (i.e., years, months) needed to recover the initial investment and break even. 2.6Additional Assessments 2.6.1Architecture Impact Assessment Describe how the investment program supports the FAA Enterprise Architecture and whether it supports the Federal Enterprise Architecture. 2.6.2Human Factors Assessment Analyze the full range of human factors and interfaces (e.g., cognitive, organizational, physical, functional, environmental) necessary to achieve an acceptable level of performance for operating, maintaining, and supporting a product in concert with meeting functional requirements. The analysis provides information on what is known and unknown about human- system performance risk in meeting minimum performance requirements. 2.6.3Information Security Assessment Security planning proceeds in parallel with system development to ensure that IT security requirements and life cycle costs for the investment are identified and validated. New initiatives that have not received a final investment decision by the Investment Decision Authority (IDA) should provide information about plans to complete security activities and documentation, which regulations will be complied with, and which methodologies will be used as appropriate. If exact dates for completion of security activities or documentation cannot be provided, an estimated date for when the information will be available should be provided. Business Case Analysis Guidelines January 6, 2010 15
  • 16. 2.6.4Safety Assessment ATO-S, Safety Engineering, provides independent concurrence that safety-related items on the IDA readiness checklist have been accomplished. Safety-related checklist items apply to the investment analysis readiness decision (IARD), the IID, the FID, and baseline change decisions. This independent concurrence requires that safety tasks and artifacts (such as safety risk- management decision memos, comparative safety assessments, and safety assessment reports) applicable to the decision being sought be brought to the ATO system safety working group for review and approval. Following review and approval, they are sent to the ATO-S Safety Engineering group. 3.0QUALITY ASSURANCE All Business Cases must be complete, reasonable, and accurate, and comply with policy, guidance, and standards. The QA process, which is part of the analysis, and formally conducted at the completion of each program’s analysis, is intended to achieve the following outcomes: • IDA decisions are supported by data-driven BCA to the maximum extent possible, • BCA products are developed using consistent governmental standards and policy values, • The business case is objective and supportable, • All interdependent program impacts associated with the acquisition are considered, • Costs, benefits, and risks are incorporated into the program baseline, and • The BCA is fully coordinated with all key stakeholders.
  • 17. APPENDICES Business Case Analysis Guidelines January 6, 2010 17
  • 18. APPENDIX A: BUSINESS CASE REQUIREMENTS Business case analysis is tailored by Acquisition Category (ACAT) and investment decision point. Business case requirements are summarized in Table A-1. Table A-1. Business Case Requirements BCA ACAT Category Initial Business Case Analysis Final Business Case Analysis Cost • Full Risk-Adjusted Life Cycle • Full Risk-Adjusted Life Cycle Cost Analysis Cost Estimates for each Estimate for Preferred Alternative alternative • At least FAA WBS Level 3 • At least FAA WBS Level 2 • Full Cost Basis of Estimate 1/2 • Parametric Estimating and Cost documentation required Factors acceptable • Bottoms-Up Estimating • SIR Cost Proposals required Benefit • Full Risk-Adjusted Life Cycle • Full Risk-Adjusted Life Cycle Analysis Benefit Estimates for each Benefit Estimates for Preferred alternative Alternative • Parametric Estimating, • Full Benefit Basis of Estimate Extrapolation and Analogies are documentation required acceptable • Program Specific Data Analysis required (i.e. simulations) Risk • Risk Applied at WBS level • Risk Applied at WBS level Analysis • Requires Monte Carlo Simulation • Requires Monte Carlo Simulation • Risk Mitigation Strategies • Risk Mitigation Strategies included in included in LCCE LCCE Schedule • Top Level Schedule Analysis • Detailed Resource Loaded Analysis Schedule Analysis Economi • Probabilistic Analysis required • Probabilistic Analysis required c • 80% confidence level • 80% confidence level Analysis Cost • Full Risk-Adjusted Life Cycle • Full Risk-Adjusted Life Cycle cost Analysis Cost Estimates for each Estimate for Preferred Alternative alternative • At least FAA WBS Level 3 • At least FAA WBS Level 2 • Full Cost Basis of Estimate 3 • Parametric Estimating and Cost documentation required Factors acceptable • Bottoms-Up Estimating • SIR Cost Proposals required Benefit • Full Risk-Adjusted Life Cycle • Full Risk-Adjusted Life Cycle Analysis Benefit Estimates for each Benefit Estimates for Preferred alternative Alternative • Parametric Estimating, • Full Benefit Basis of Estimate Extrapolation and Analogies are documentation required acceptable • Program Specific Data Analysis required (i.e. simulations) Risk • Risk Applied at WBS level • Risk Applied at WBS level Analysis • Does not require Monte Carlo • Requires Monte Carlo Simulation Simulation • Risk Mitigation Strategies included in LCCE
  • 19. Schedule • Top-Level Schedule Analysis • Top-Level Schedule Analysis Analysis Economi • Deterministic Analysis • Probabilistic Analysis Required c Acceptable Analysis BCA ACAT Category Initial Business Case Analysis Final Business Case Analysis Cost • Full Risk-Adjusted Life Cycle • Full Risk-Adjusted Life Cycle Cost Analysis Cost Estimates for each Estimates for Preferred Alternative 4/5 alternative • At least FAA WBS Level 2 (New • At least FAA WBS Level 2 • Parametric Estimating and Cost Function- • Parametric Estimating and Cost Factors are acceptable ality) Factors acceptable • Market Surveys or SIR results required Benefit • Full Risk-Adjusted Life Cycle • Full Risk-Adjusted Life Cycle Analysis Benefit Estimates for each Benefit Estimates for each alternative alternative • Parametric Estimating, • Parametric Estimating, Extrapolation Extrapolation and Analogies are and Analogies are acceptable acceptable Risk • Risk Applied at WBS level or • Risk Applied at WBS level or Analysis Bottom Level Bottom Level • Does not require Monte Carlo • Does not require Monte Carlo Simulation Simulation Schedule • Does not require additional • Does not require additional Analysis Schedule Analysis beyond Schedule Analysis beyond Implementation Strategy and Implementation Strategy and Planning Document (ISPD) Planning Document (ISPD) requirements requirements Economi • Deterministic Analysis • Deterministic Analysis acceptable c acceptable Analysis Cost • Cost-Effectiveness Analysis Only 4/5 Analysis • Full Risk-Adjusted Life Cycle Cost (Tech Estimates for Preferred Alternative Refresh • Initial Investment Analysis (IID) • At least FAA WBS Level 2 or not required • Parametric Estimating and Cost Replace- Factors acceptable ment-in- • Market Surveys or SIR results Kind) required Benefit • Cost avoidance analysis and denial Analysis of operational capability/service assessment acceptable Risk • Risk Applied at WBS level or Analysis Bottom Level • Does not require Monte Carlo Simulation Schedule • Does not require additional Analysis Schedule Analysis beyond Implementation Strategy and Planning Document (ISPD) requirements Business Case Analysis Guidelines January 6, 2010 A-19
  • 20. Economi • Does not require NPV, or Payback c Analysis Analysis
  • 21. APPENDIX B: BUSINESS CASE TEMPLATE Business Case Analysis Guidelines January 6, 2010 B-21
  • 22. Federal Aviation Administration FINAL January 6, 2010 Final Business Case for (Name of Proposed Investment) Approved By: ________________________________________Date:_____________ Name, Title, Originating Office
  • 23. SIGNATURE PAGE Business Case for (Name of Proposed Investment) Enterprise Architecture Roadmap Statement # Approved by: Date: Vice President or Director of Sponsoring Service Organization or Line of Business Focal Point Name Organizational Code Phone Number FAX Number Federal Aviation Administration 800 Independence Avenue SW Washington, DC 20591 Business Case Analysis Guidelines January 6, 2010 B-23
  • 24. Table of Contents 1.0 Executive Summary 2.0 Problem Statement 3.0 Assumptions, Constraints, Conditions 4.0 Current State Description – “Status Quo” or “As Is” 5.0 Future State Description – “To-Be” 6.0 Alternatives Analyses and Metrics 6.1 Life Cycle Cost 6.2 Benefits (quantitative/non-quantitative) 6.3 Schedule 6.4 Risk and Sensitivity 6.5 Economic 6.5.1 Cost/Benefit analysis 6.5.2 Net Present Value (NPV) 6.5.3 Payback Period 6.6 Related Assessments 6.7 Budget Impact 6.8 Funding Profile 7.0 Recommendation(s)
  • 25. 1 Executive Summary Summarize the key information in this document, highlighting those elements that should be most relevant for the Investment Decision Authority to make an investment decision. 2 Problem Statement Identify the problem this initiative is intended to address, or projected shortfall that needs to be cured in order to meet FAA needs. Reference the documents produced during Concept and Requirements Definition for this investment initiative and the Enterprise Architecture. 3 Assumptions, Constraints, and Conditions Briefly describe important assumptions, constraints, and conditions having major influence on the analysis and its conclusions. The following should be considered as a minimum: the assumed remaining service life of the currently fielded capability, the assumed required operational date for the proposed investment, the assumed economic service life of any proposed investment, and the operational framework within which any new investment must function. 4 Current State Description – “Status Quo’ or “As Is” Describe the set of existing or anticipated conditions if policies, configurations, regulations, and management strategies remain unchanged. Describe in detail the current system capabilities that will be addressed by the planned initiative. Describe impacts to current operations from an end- user perspective. 5 Future State Description – “To Be” Describe the future capabilities needed to support the latest approved Enterprise Architecture Roadmap. Describe impacts to operations from an end-user perspective. 6 Alternatives Analysis and Metrics List and briefly describe the alternatives analyzed as part of the BCA. Summarize the evaluation criteria and their relative weighting used in evaluating each alternative. 6.1 Life Cycle Cost Briefly summarize the cost analysis conducted to generate high-confidence life cycle cost estimates for each alternative. Include the life cycle cost estimate for each alternative. Cost estimates shall be presented in base-year and in then-year dollars. Summarize sensitivity analysis performed on cost drivers and the effect of technical, schedule and cost risk on program cost and schedule. Business Case Analysis Guidelines January 6, 2010 B-25
  • 26. 6.2 Benefits Analysis (quantitative/non-quantitative) Summarize benefits analysis activities conducted to generate high-confidence life cycle benefit estimates for each alternative. Include the life cycle benefit estimates for each alternative. For an initial investment decision, the summary should provide enough information to show the differentiation and relative merits of each alternative. For the final investment decision, the summary should provide more detailed, possible site-specific information, based on a greater definition and understanding and less uncertainty about the proposed investment. Include at a minimum the ground rules and assumptions, relevant measures for the program, a description of the pre-investment baseline, quantified and non-quantified benefits impacts, FAA benefits, user benefits, and an overall summary of benefits. 6.3 Schedule Analysis Provide general assumptions regarding the schedule for each alternative. Summarize the schedule analysis results of each alternative or the proposed investment program schedule. The summary should include a discussion of activities taken to generate risk-adjusted schedules for each alternative or the proposed investment program schedule. 6.4 Risk and Sensitivity Analysis Summarize risk-assessment activity and results. Include the overall risk ratings for each alternative or for the proposed investment program. Identify major risks and planned mitigation strategies for each alternative. Discuss the comparative risk assessment for the initial investment decision. 6.5 Economic Analysis Summarize the cost and benefit analysis conducted for each alternative or proposed investment. Include benefit-cost ratio, net present value, and payback period as a minimum. 6.5.1 Cost/Benefit Analysis For each alternative, weigh the total expected costs against the total expected benefits. Include the designated discount rate per OMB guidelines to calculate the present-value cost and benefits based on risk-adjusted estimates for benefits and costs. 6.5.2 Net Present Value (NPV) For each alternative show the monetized value of a future stream of expected net benefits discounted to the present by the current desired rate of return. 6.5.3 Payback Period Show assumptions and calculations used to determine the length of time required to recover the cost of each alternative. 6.6 Related Assessments Summarize activities and results of the architecture impact assessment, human factors
  • 27. assessment, information security assessment, and safety assessment. 6.7 Budget Impact The Capital Investment Team assesses the budget impact of each alternative against other investment programs in the FAA's financial baseline and their relative contribution to satisfying goals in the strategic plan. When an alternative solution cannot be funded within the capital investment baseline, the Capital Investment Team may propose offsets from lower priority programs. Preliminary budget impact assessments by the Capital Investment Team shape subsequent deliberations of the BCA team. 6.8 Funding Profile Display the funding profile for each alternative by fiscal year in then-year dollars. 7. Recommendation Identify the recommended alternative, if any, and summarize the rationale for the recommendation. Appendix A: Business Case Analysis Team members. Identify team members and their organizations. Define briefly the role of each in table format Appendix B: References List documentation used as references in conducting the BCA. Examples of reference documentation include basis of estimates for cost, benefits, and schedule and documentation of the related assessments. Each reference should include documentation title, originating organization, and date. Business Case Analysis Guidelines January 6, 2010 B-27
  • 28. APPENDIX C: HELPFUL WEBSITES Life Cycle Costs For additional cost estimating and business case guidance: http://atoexperience.faa.gov/finance/index.php? option=com_content&task=view&id=172&Itemid=97 • ATO F&E to OPS Transition • Guidelines for Evaluating FAA Cost Estimates • Independent Evaluation Template • Questions Executives Ask GAO Cost Estimating and Assessment Guide, Chapter 4 GAO Cost Estimating and Assessment Guide: http://www.gao.gov/htext/d093sp.html This is the result of several years of work and coordination lead by GAO and using both government and private sector expertise. It provides detailed guidance on how to prepare cost estimates, use of WBS, data collection, etc and is supported by numerous examples. It has been adopted by the FAA as its estimating standard. Cost Basis of Estimate: http://fast.faa.gov/investment/cboe.htm Provides useful advice in Sections 3 and 4 on what information is required and how to document cost estimates. Cost Estimation Methodology: http://fast.faa.gov/investment/scem.htm FAA guidance on how to prepare cost estimates and is useful for its FAA specific information. It should be used in conjunction with the GAO guide which is a broader and more thorough look at the cost estimating process. GAO Cost Estimating and Assessment Guide: http://www.gao.gov/htext/d093sp.html This is the result of several years of work and coordination lead by GAO and using both government and private sector expertise. It provides detailed guidance on how to prepare cost estimates, use of WBS, data collection, etc and is supported by numerous examples. It has been adopted by the FAA as its estimating standard. For additional information on how to prepare an IGCE: Cost and Price Methodology: http://fasteditapp.faa.gov/ams/do_action? do_action=LinkSection&contentVersionUID=10042&contentUID=3&sectionNumber=0.22.1.2 This site provides guidance on how to prepare an IGCE.
  • 29. Chief Financial Officer’s web site https://intranet.faa.gov/faaemployees/org/staffoffices/aba/oversight/ OMB Circular A-11, Preparation, Submission and Execution of the Budget, Part7, Planning, Budgeting, Acquisition and Management of Capital Assets and Supplement to Part 7, Capital Programming Guide: http://www.whitehouse.gov/omb/circulars_a11_current_year_a11_toc/ The Capital Programming Guide provides guidance for a disciplined capital programming process, as well as techniques for planning and budgeting, acquisition, and management and disposition of capital assets. IDA Guidance: http://atoexperience.faa.gov/ipm/index.php? option=com_content&task=view&id=57 This link provides IDA templates for IID decisions. For more WBS information: Standard FAA WBS and definitions http://fasteditapp.faa.gov/ams/do_action?do_action=ListTOC&contentUID=5 GAO Cost Estimating and Assessment Guide, Chapter 8: http://www.gao.gov/htext/d093sp.html The guide provides a thorough description, use, and best practices for developing and using a WBS. For additional parametric cost estimating guidance, see: http://atoexperience.faa.gov/finance/ index.php?option=com_content&task=view&id=172&Itemid=97 Pocket Estimating Guide (PEG) F&E Factors Pocket Estimating Guide (PEG) Ops Factors Benefits Benefit Basis of Estimate: http://fast.faa.gov/investment/DSFAABE.htm This describes in detail the necessary elements for writing a benefits basis of estimate, which documents the processes and the methodologies utilized to produce the benefits estimate. Benefit Analysis Methodology: http://fast.faa.gov/investment/bam.htm Provides guidelines and processes for preparing benefits estimates. It includes the necessary elements for writing a benefits basis of estimate, which documents the processes and the methodologies utilized to produce the benefits estimate. Business Case Analysis Guidelines January 6, 2010 C-29
  • 30. Information and insights for estimating benefits http://atoexperience.faa.gov/finance/index.php? option=com_content&task=view&id=172&Itemid=97 The Art of Benefits Estimation Guidelines for Quantifying Environmental Benefits Basic Guide to Human Factors Risk Analysis Risk Analysis Basis of Estimate: http://fast.faa.gov/investment/sbera.htm Provides the principles of risk analysis. Note still refers to JRC 1 and JRC 2, but the information is useful. Risk Metrics for Initial Investment Analysis: http://fast.faa.gov/investment/rmiia_jrc2a.htm Note still uses outdated terms (JRC 2A), but provides useful information discusses risk metrics and how to prepare a risk analysis. Risk Analysis Methodology for Initial Investment Decision: http://fast.faa.gov/investment/giatara.htm How to conduct a risk assessment for initial investment decision. Risk Analysis Methodology for Final Investment Decision: http://fast.faa.gov/investment/iargfid.htm How to conduct a risk assessment for a final investment decision. Schedule GAO Cost Estimating and Assessment Guide, Chapter 4: http://www.gao.gov/htext/d093sp.html GAO Cost Estimating and Assessment Guide, Appendix X, Schedule Risk Analysis discusses how to prepare a risk-adjusted schedule. Economic Analysis Economic Factors and Analysis: http://atoexperience.faa.gov/finance/index.php? option=com_content&task=view&id=172&Itemid=97 An FAA document containing inflation rates, discount rates, federal labor rates etc. that should be used in analyses supporting FAA projects. Discount Rate Guidance: http://www.whitehouse.gov/omb/circulars_default/ OMB Circular A-94, "Guidelines and Discount Rates for Benefit-Cost Analysis of Federal Programs" (10/29/1992) provides guidance and discount rates to be used for analysis.
  • 31. APPENDIX D: LIST OF ACRONYMS Acronym Full Name ACAT Acquisition Category AEB Acquisition Executive Board AIO Assistant Administrator for Information Services AJF-3 Office of Investment Planning and Analysis AMS Acquisition Management System ATO Air Traffic Organization B/C Benefit – Cost Ratio BOE Basis of Estimate CFO Chief Financial Officer CIO Chief Information Officer (AIO-1) CIT Capital Investment Team CRD Concept and Requirements Definition CTO Chief Technology Officer DOD Department of Defense DOT Department of Transportation EA Enterprise Architecture EAB Enterprise Architecture Board EC Executive Council FAA Federal Aviation Administration F&E Facilities and Equipment FID Final Investment Decision fRD final Requirements Document GAO Government Accounting Office IA Investment Analysis IAP Investment Analysis Plan IARD Investment Analysis Readiness Decision IDA Investment Decision Authority IGCE Independent Government Cost Estimate IID Initial Investment Decision IRR Internal Rate of Return ITEB Information Technology Executive Board JRC Joint Resource Council LCCE Life Cycle Cost Estimate LOB Line of Business NAS National Airspace System NPV Net Present Value OMB Office of Management and Budget PEG Pocket Estimating Guide POC Point of Contact pRD preliminary Requirements Document QA Quality Assurance RD Requirements Document SME Subject Matter Expert WBS Work Breakdown Structure Business Case Analysis Guidelines January 6, 2010 D-31