Software Cost Contingency Development


Published on

Point of View on Developing Software Project Cost Contingencies

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Software Cost Contingency Development

  1. 1. D. B. Skillern – Delivery Excellence Manager 2010 February 14 Point of View on Developing Software Project Cost Contingencies © 2010 IBM Corporation
  2. 2. Over a quarter of project failures are due to budgeting and planning errors Analysis completed by a variety of experts as well as anecdotal experience has shown that the typical project cost contingencies are inadequate. For budgeting purposes, the authorized project cost for most software development projects include some element of a cost contingency. The typical contingency is based on internal policy and approval guidelines. Experience has noted that the range of variance to a single project estimate on any project at the early stages of the development life cycle (e.g., feasibility and requirements definition) can lie in a wide range well beyond the typical contingency. The statistical range for variance from the initial estimate of a software project cost ranges from 60 percent higher than the estimate to 30 percent lower than the original estimate. The large range of differences between estimated costs and actual costs for IT projects was first identified in a study done by Barry W. Boehm, in 1981 “Software Engineering Economics1”. This analysis has been reviewed many times by other experts both in the software industry and academia since publication. IBM has confirmed and further extended the findings of Mr. Boehm’s analysis. In recent studies done by The Gartner Group, project failures range from 20% to 40% of all IT projects depending on size1. Even at companies that aren’t among the top 25% of technology users, three out of ten IT projects [on average] fail. Translation: an amazing 30% experience project failures.2 Over a quarter of these failures were due to significant budgeting and planning errors. 2 © 2010 IBM Corporation
  3. 3. Typical cost contingencies are often inadequate The typical budget for an IT project includes cost contingency. The cost contingency is based on internal guidelines. The guidelines usually limit the contingency to 10% to 15% of the budgeted cost estimate. The percentage contingency factor is designed to limit cost over-runs to minor scope changes and estimate differences, before supplemental approval is required. Based on academic analysis and practical experience, this level of cost contingency is often inadequate to absorb large estimate variances that occur. However, there are approaches that can limit the risk of cost over-runs. These approaches include use of a packaged software solution (aka, Commercial Off- the-Shelf Software), application of proper estimation techniques, installation of a sound project management control system and deployment of a skilled project manager. While these approaches can reduce the amount of contingency needed, the requirement for cost contingency is not eliminated. The following describes approaches to improve cost contingency estimation and reduce potential over-runs to the budget project cost estimate. 3 © 2010 IBM Corporation
  4. 4. Realize that all software project estimates are approximations All estimates of software development costs are approximations. The estimate is an educated guess of what might happen in the future, expressed with a stated degree of confidence (i.e., potential variance), for a specific scope of work. The degree of confidence that the estimator has in the estimate, as expressed in the variance, depends on the techniques used in the estimate process and the knowledge of what is required. The variance around the cost estimate measures the potential range of results that can be expected. Basically, this is how much actual results can differ from estimated results. The degree of confidence in the accuracy of a particular estimate (and potential estimate variance) depends on the amount of uncertainty existing when the estimate is developed. Typically, early in a project life-cycle, considerable uncertainty exists around the requirements for the project. Extensive analysis has been completed regarding the potential range of variance for a software development project over the development life-cycle. The analysis indicates that while there may be different statistical distributions of variance around the estimate, a variance, none-the-less, will exist. There are certain actions that can be taken to mitigate the impact of the variance. The actions involve the project management control system deployed, the estimation techniques used, the project manager assigned and the cost contingency maintained. The following discusses estimate variance in software development projects and the possible actions that can be taken to mitigate the impact of such variance. 4 © 2010 IBM Corporation
  5. 5. Understand that estimate differences will occur Estimation accuracy in software development is a problem that vexes every project and program manager. The topic has been studied extensively. Different models have been developed for software development estimation – ranging from statistically based mathematical calculations to judgment based analogies. Each type of estimation technique has inherent limitations. The various studies done on the accuracy of software development cost estimation have reached similar fundamental conclusions. Regardless of technique used to develop the estimate, a variance will exist for the estimated effort and cost of the project. The variance range will narrow depending on the knowledge of what is needed and how the work is completed. Essentially, estimate accuracy and confidence will improve as more is known and uncertainty is reduced. Customer Acceptance Defects by Effort Hours Development Hours Required By IT Organization CMMI Level 250 9000 8000 200 7000 CMMI level 1 6000 150 5000 4000 100 3000 CMMI level 4 2000 50 1000 NB Shows Linear and Order 2 polynomial trend lines. 0 0 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 0 2000 4000 6000 8000 10000 12000 14000 16000 Source: Software Engineering. A Practitioners Approach, Roger S. 5 Source: IBM Analysis, 2008. Pressman, Pressman, ISBN 0-07-707936-1. © 2010 IBM Corporation
  6. 6. Recognize that the cost differences (variance) can be large The original analysis of software development cost variance was completed by Mr. Barry W. Boehm in 19813. The analysis was based on certain Department of Defense information. The analysis considered software project cost estimates as a function of the life-cycle phase of the project. This function has been empirically proven in further completed by Mr. Luiz A. Laranjeira, in “Software Estimation of Object- Oriented Systems4”. The work has also been reviewed and validated by various other academicians and practicing consultants, including IBM since its initial release. Historical experience with software development cost variances indicate that larger variances exist for software development efforts and smaller variances will occur for simple package software installations. The software development cost variance analysis (based on custom development) are depicted below: 3,4 6 © 2010 IBM Corporation
  7. 7. Be thorough in building the original cost estimate In addition to the complexity of the project driving the need for a higher contingency to address estimate variance, the quality of the original estimate will impact the amount of cost contingency required. Accordingly, the contingency for high quality estimate will be lower than the contingency on a quick back of the envelope number, so it is critical that the estimate process be thorough. There are an number of simple rules5 that can be applied to improve the estimate quality. Some examples of these rules follow: – Use two or more estimating approaches, viewing the problem from different angles improves the accuracy and the confidence in the estimate. Three example methods are: the analogy, metrics based and duration based models (with time constraints). There are inherent limitations in all estimation approaches, but the limitations associated with each method can be understood and considered in the process. – Examine the requirements and translate into a scope of work within a given solution with specific measurements. The scope can be quantified to reflect the effort or cost drivers that influence work to be done. Typically such quantification involves 'counts' of work efforts: number of prototype sessions, number of core business processes, number of reports, number of interfaces, etc. – Use an estimate model when possible, software packages can be purchased to complete complex estimate development for custom software projects (e.g., COCOMO), but for a estimating the effort to implement an accounting software package (e.g., Great Plains) a spreadsheet will suffice. – Use the expertise of person experienced in developing the particular software in the project. Experience build on many projects will improve the quality of the estimate. In software development, the best predictor of future performance is how it was done in the past. Supporting an estimate with past metrics removes subjectivity. Also recognize that estimates developed by software sales people may be biased due to their desire to make a sale software sale. – Recognize what you don’t know. Identify the key assumptions and risks factors associated with the project. These elements represent elements to be controlled to minimize the variance to the cost estimate. 7 © 2010 IBM Corporation
  8. 8. Understand what do know and how you can control it What you don’t know about project definition (requirements, complexities, scope, resource skills, etc.) represents uncertainty that drives estimate variances. What you know you should control. What you don’t know represents risks you should mitigate. Recent analysis completed by Mr. Murray Cantor, an IBM Distinguished Engineer, has indicated that there may be multiple probability distributions impacting project estimates. Mr. Cantor’s work, summarized in an IBM technique paper6, generally validated the work of Mr. Boehm. Essentially, at the outset of a project, very little is known for sure – the effort duration and expected cost are, at best, educated guesses. In mathematical terms, the factors and values of an estimate are made up of known quantities (such the software product to be used) and random variables (such as the changes need to the software product to meet requirements). The cost of a project will vary depending on the number of random variables and the possible values of these random variables. The analysis of Mr. Cantor supports the position that certainty increases around these project parameters as time passes. The amount of time to complete certain tasks becomes known, so the estimate variance (and range in the probability distribution) for the estimate and required contingency decreases. The analysis suggests that many different statistic distributions can exist for the project parameters (normal-bell curve, log-normal, triangular, etc.). Even though many statistical distributions exist depending on the specific facts and circumstances of a project, it is critical to recognize that project cost variance will exist and techniques need to be applied to control the variance. Developing a cost contingency recognizes the impact of limitations to mitigating risks that drive estimated cost variances. 8 © 2010 IBM Corporation
  9. 9. Identify a realistic cost contingency Build a cost contingency for a project that recognizes the Risk Drivers risks associated with the project. Realize that the size of Organization Change Level Organization Priority 80 Package Softw are Custom izations this contingency may be larger than the typical business Technology Platform 70 Softw are Integration control contingency percentage. The business control User Readiness 60 50 Staff Capabilities contingency cap approach is designed to avoid the need SME Availability 40 30 Project Manager for supplemental approvals for minor variances and Data Quality 20 10 Project Managem ent System properly manage funding requirements. The alternative 0 Decision Making Speed Technical Com plexity to managing with a larger cost contingency is to supplement with additional authorized costs as cost over-runs when risks become realized project issues. Leadership Support Size and Breadth Either approach requires proper expectations Tim eline Constaints External Requirem ents management. Softw are Developm ent Maturity In developing the right contingency for a project, the risk profile should be considered. The potential risks should be scaled to model the potential risk contingency required. The scaling depends on risk assessment (probability of occurrence, impact of risk, etc.) for a particular effort and how well these risks can be mitigated. After the risk contingency estimate is determined, evaluate the business case with the worst case risk scenario to determine what is acceptable. The risk contingency can also be measured in mathematically terms through calculation of variance using more complex estimation techniques such as the PERT technique or the Delphi method. The final amount of contingency maintained for any project is a matter of judgment and philosophy as well as the internal practices of the organization involved. The critical factor is to recognize that a contingency will be needed for any project effort. Based the contingency on known risks and minimize the use of that contingency with an effective project management control system. 9 © 2010 IBM Corporation
  10. 10. Understand that a sound project management control system reduces inherent uncertainty that a savvy project manager can not do alone Project cost contingencies will be required to address project risks. Project Management Success Effectively managed projects tend to face lower variances to the original cost estimate (and use less contingency) for a project. Effective project Highly Skilled management execution depends on both: Possible Highly - the skills of the project/program management team and Success Probable Success Manager Project Skills - the quality of the project management control system. A sound Project Management Control System (PMCS) represents the tools and techniques to manage and mitigate risks that drive cost Success Possible estimate variance. The risks for any project arise in a number of areas. Unlikely Success One area of risk is the program will not achieve the cost and schedule Limited Experience targets. A typical PMCS includes certain control activities focused on Poor Quality High Quality these objectives. Example PMCS activities include Project Governance, Work Plan Management and Risk Mitigation provide tasks to reduce Project Management Control System uncertainty and cost variance. Structure The key to reducing the cost variances is to reduce the uncertainty. “Even if, at the onset of the project, the initial variance is high, the project could be managed so that early in the project, the uncertainties are removed so that the variance is low. In fact, the key to project success is removing the uncertainty as soon as possible, resulting in the increased ability to make accurate and aggressive plans. However, since the ability to reduce variance is based on knowledge gained through project experience,”6 only in the most simple projects can the uncertainties be removed quickly. Control what you can and plan for what you can’t control. Plan for the uncertainty inherent to software development and implementation projects. Building a quality project cost estimate supported by realistic project contingencies, a sound project management controls and a qualified project manager are proven steps to address uncertainty and the risks of project failure. 10 © 2010 IBM Corporation
  11. 11. Notes: 1 Gartner Group Studies, including “The User’s view of Why IT Projects Fail”, 2003, 2004, and 2005. 2 “Survey shows common IT woes”, Julia King, Computerworld, June 2003 3 Software Engineering Economics, Barry W. Boehm, Prentice Hall PTR, 1981 4 “Software Size Estimation of Object-Oriented Systems”, IEE Transactions on Software Engineering, Vol. 16, No.5, May 1990 5 “Estimating – Top Ten Best Practices”, Estimating at IBM, October 27, 2009 6 ”Estimate variance and governance”, IBM Developer Works, March 15, 2006 11 © 2010 IBM Corporation