7 Signs of Bad Project Estimates - And How You Can Fix It

10,303 views

Published on

This presentation focuses on root causes of terrible estimating in software projects. In addition to recommending best practices, this presentation demonstrates - using live project history data - how unenlightened management actually causes the estimate and project failures. These are management - not technical - problems.
You view and hear audio at www.owsoftware.com/demo/7signs/7-Signs-You-Have-a-Bad-Estimate-081112.mp4
(Note: it may take a while to buffer before it starts running)

Published in: Business, Technology

7 Signs of Bad Project Estimates - And How You Can Fix It

  1. 1. 7 Signs You Have a Bad Project Estimate (And what to do about it) January 20th, 2010 Kolinger Associates, LLC 2010 All Rights Reserved
  2. 2. Today’s Presentation Estimating project cost and schedule is a serious challenge to our companies. Common symptoms include projects that refuse to end, projects cancelled, people overworked, missing a window of opportunity for a new product, and overruns causing an inability to fund other important initiatives. Tools may help - but this presentation is primarily about the ideas and concepts necessary for improvement. Kolinger Associates, LLC 2010 All Rights Reserved
  3. 3. Overview • This presentation considers factors that are common to most but some specific references to software development estimation. • Me: 25+ years, mostly in large corporations. Developed software, consulted, trained, managed – and recovered - scores of projects. • Founded OfficeWork Software in 2005. Kolinger Associates, LLC 2010 All Rights Reserved
  4. 4. Bad Estimates Very Common A “Bad Estimate” working definition: Money got spent and the required result was not achieved in time. A few sundry findings: • 67% of all projects under-planned by 290% • 1% of projects meet time, budget, and scope goals • 60% of all projects over 50k Lines of Code cancelled before completion. • Common gripe: “<<your organization>> does not deliver” (82.3% of all statistics are made up on the spot. Just kidding.) Kolinger Associates, LLC 2010 All Rights Reserved
  5. 5. The Saga 1. Sufficient assets are not allocated, and there is not enough time. 2. The schedule (if there is one) starts to slip, one day at a time. 3. The project manager suddenly realizes the slippage and seeks a culprit. 4. People from various departments involved accuse people from other departments of delaying the project. More time is wasted in finger-pointing. 5. To make up time, the project manager decides to “crash” the project by applying more assets/resources to all activities that are currently being performed. 6. Everyone scrambles to crash his job, and people are infuriated to find out that they are either further behind or finished and their part is not yet needed. Interest wanes as people chafe under the new delays. The project is over-budget due to all the crashing, and the whole thing is either: – a. on-time but shoddy, – b. well-done but late, – c. both shoddy and late, or – d. abandoned. Dr. Robert Graham – Project Management as if People Mattered Kolinger Associates, LLC 2010 All Rights Reserved
  6. 6. Question Think of a recent project, that went over budget, was late, or fell short of the original scope. – Why did that happen? – What could you have done about it? (Write down your answer) Kolinger Associates, LLC 2010 All Rights Reserved
  7. 7. Test Your Estimation IQ Question: Why do IT Project Managers estimate at the drop of a hat? Answer: Genetic Mutation! • Have Project • Will Estimate • No Requirements Needed! Actual Project Manager DNA ! Warning: Do not ride in the elevator with your project sponsor! (Hint: You might rattle off an estimate!) Gopal Kapur, Center for Project Management Kolinger Associates, LLC 2010 All Rights Reserved
  8. 8. Question? 1. How many of you have managers who pressure for cost and date estimates before requirements are clear? 2. How many of you are those managers? 3. What are you doing to encourage better estimating? 4. How is it going? Kolinger Associates, LLC 2010 All Rights Reserved
  9. 9. Let’s Estimate! The Project (Input): Web-based information system. (Legacy system already exists; let’s just make it web-based) The Estimation (Magic wand) Process The Estimate (Output): – 12 months – 4.4 FTE – 5,600 hours – $400,000 “We will have it for you in 12 months” Kolinger Associates, LLC 2010 All Rights Reserved
  10. 10. What to look for So, how can you recognize the bad estimate… before you get bit? 12 months 4.4 FTE 5,600 hours $400,000 Kolinger Associates, LLC 2010 All Rights Reserved
  11. 11. 1. When You Hear ‘Done’ More Than 4 Times. So, what the heck does ‘done’ look like? Look for a clear, written definition of the end-result. •Project ‘success’ is meeting expectations. Period. • Common finding: – Everyone ‘knows’ – Nobody agrees •Define the project – in writing - one-pager: – Description: (What, where, when, how much) – Desired Results: (Benefits that will be achieved) – In Scope / Out of Scope – Assumptions •De-babelize –Use language everyone understands •Sign-off from the sponsor and each stakeholder is critical. (Harder than it sounds) Kolinger Associates, LLC 2010 All Rights Reserved
  12. 12. 2. When you Hear, “What is the Plan?” The estimate is based on the plan. Look for the plan with evidence of traceability to the project definition and make sure everyone is aware. If you ever hear people saying, “I don’t know what the plan is”, take action!! Your job as project manager is to keep everyone honest to the project’s schedule, scope, and budget. Even if the estimate were ‘perfect’ you can render it useless through a poor plan. • The plan includes, but is more than the project schedule • The plan includes the project definition, the people, a WBS (Work Breakdown by deliverable), estimates, resource assignments • Milestones and activities are connected (or linked) so that a critical path can be identified and managed. • Connections are critical as they indicate hand-offs from one person to another. • Each deliverable has an owner • Each person on the team is associated with a deliverable • Have an independent group review the plan. (Find bug on paper vs. production). Kolinger Associates, LLC 2010 All Rights Reserved
  13. 13. Clue: Missing Dependencies In this case, ‘date’ was the primary goal Kolinger Associates, LLC 2010 All Rights Reserved
  14. 14. View resource graph and Gantt Kolinger Associates, LLC 2010 All Rights Reserved
  15. 15. Review usage in excel Kolinger Associates, LLC 2010 All Rights Reserved
  16. 16. 3. When You Have a Single Number Estimate Look for a schedule based on appropriate estimate ranges Two critical lessons: 1. Projects are bigger than they appear. 2. The road to project hell is paved with single number estimates. Gopal Kapur, Center for Project Management Best Case Most Likely Worst Case Software is “different” Kolinger Associates, LLC 2010 All Rights Reserved
  17. 17. Estimating Confidence in Software Delivery Effort Duration Project cost Observed Limits of Estimate Accuracy Project (effort and size) schedule + 400 % 4x 1.6x Overestimated 2x 1.25x Completion 1.5x 1.15x 1.25x 1.1x Exact 0% 1.0x 1.0x Fair 0.8x 0.9x 0.67x 0.85x 0.5x 0.8x Rough Underestimated 0.25x 0.6x Sizing - 400% Initial Approved Requirements Product Detailed Product product product specification design design complete definition definition specification specification Best Most Likely Worst 10% 50% 90% Chance Chance Chance B. Boehm – SW Engineering Economics Kolinger Associates, LLC 2010 All Rights Reserved
  18. 18. 4. When It Has Not Been Done Before Look for an estimate that factors your capability Project management is all about making promises and keeping promises. Promises are more likely to be kept if: • Made voluntarily (you agree) • It’s your commitment (your data) • it is according to capability (i.e., you have proven ability) Include Risk Management (Project Management for Adults) To maintain a viable estimate constantly manage project adversity and team capability Stan Rifkin – Master Systems Tom DeMarco - ASG Robert Charette - ITHABI Kolinger Associates, LLC 2010 All Rights Reserved
  19. 19. 5. When You Can’t See Evidence of Historical Data Look for an estimate that uses historical performance data as a reference • The estimate should consider a reasonable work week for resources based upon historical data for individuals and the organization. • Examples: – 60% productivity or 24 hours/week/person – Individual characteristics • Technical skill factor • Business knowledge factor – Careful use of overtime – Defect and rework rates Only lunch breaks are 100% productive Kolinger Associates, LLC 2010 All Rights Reserved
  20. 20. Efficiency Kolinger Associates, LLC 2010 All Rights Reserved
  21. 21. Question • What percent of a programmer’s time is spent coding? • How is this factored into the estimate and schedule? Kolinger Associates, LLC 2010 All Rights Reserved
  22. 22. What do Programmers Do? Kolinger Associates, LLC 2010 All Rights Reserved
  23. 23. Look for a Repeatable Estimating Process Historical data is only valuable (normalized) in the context of a repeatable process Kolinger Associates, LLC 2010 All Rights Reserved
  24. 24. Estimation Methods • Formula • Activity-based • Analogy Kolinger Associates, LLC 2010 All Rights Reserved
  25. 25. How to Estimate (Kapur “Flinch” Method) $850B …. For PHASE 1 ! Kolinger Associates, LLC 2010 All Rights Reserved
  26. 26. Balancing the Estimate Your Goal: Commitment according to capability! Kolinger Associates, LLC 2010 All Rights Reserved
  27. 27. Project Difficulty vs. Management Capability Project: Web-Enabled Business Information System Kolinger Associates, LLC 2010 All Rights Reserved
  28. 28. Historical Data: Example of Estimating Maturity Effort Duration Project cost Observed Limits of Estimate Accuracy Project (effort and size) schedule + 400 % 4x 1.6x Overestimated 6. Completion 2x 1.25x 1.5x 1.15x 1.25x 1.1x Exact 0% 1.0x 1.0x 0.8x 0.9x 0.67x 0.85x 0.5x 0.8x Underestimated 0.25x 0.6x - 400% Initial Approved Requirements Product Detailed Product product product specification design design complete definition definition specification specification Legend Idea / OP H/L Requirements Most Code Complete Project Complete - Effort / Budget 10/1/02 9/1/03 3/1/04 8/2/04 - Schedule ~ 392% Variance ~ 317% Variance Requirements ~ 33% Variance FTE 4.4 FTE 4.4 Complete *1 FTE 13.4 FTE 17.9 2/1/04 ~ 80%Variance Footnote: * 1. Independent Estimate at point FTE 10.0 of most requirements complete Kolinger Associates, LLC 2010 All Rights Reserved
  29. 29. An Example of Estimating [Im]Maturity Estimation Accuracy Commercial Web System Project OP / Idea H/L Req Most Code Project Milestone (Forecast) Done Complete Complete (Forecast) (Forecast) (Actual) Date of Estimate 10/1/02 9/1/03 3/1/04 8/1/04 Final Effort Hours 5,600 unknown ~ 16,000 25,000 Cost $ 402,000 $ 475,000 $ 1,484,429 $ 1,979,238 Cost Variance 392% 317% 33% 0% FTE 4.4 4.4 13.4 17.9 FTE Variance 311% 311% 33% 0% Kolinger Associates, LLC 2010 All Rights Reserved
  30. 30. Where did the money go? Pay to put bugs in. $280k $100k $20k $560k $620k Pay to take bugs out. How much waste? ~60% Re-work! Kolinger Associates, LLC 2010 All Rights Reserved
  31. 31. Quality Results Defects Found Pre and Post Implementation 1. Defects Discovered in Testing (after “Code Freeze”): Code defects: 977 (70%) Missing / Incorrect Requirements: 394 (30%) Total: 1,371 2. Defects Discovered by the User in Production: First 90 days: 136 Kolinger Associates, LLC 2010 All Rights Reserved
  32. 32. Better Estimation via Better Control of Quality Comparing Defect Removal Results of Two Projects Case A Case B Activity (Dollars) (Person Hours) Defects (Dollars) (Person Hours) Defects Top-Level $2,875 115 50 $0 0 0 Design Review Detailed Cheap $17,500 700 140 $0 0 0 Design Review Code $42,500 1,700 110 $0 0 0 Inspection Unit Test $19,250 770 50 $38,000 1,500 125 Expensive Integration $62,250 2,490 100 $107,205 4,290 190 Test Total $128,625 5,775 450 (90%) $145,250 5,810 315 Maintenance $325,000 13,000 50 $797,500 31,900 185 Lifecycle Total $453,625 18,755 500 $942,750 37,710 500 Reference: Capers Jones, SPR Kolinger Associates, LLC 2010 All Rights Reserved
  33. 33. Question – Crash Schedule? • What happens when we compress the schedule? • Good to know when devising a plan that will work – and some exec asks for it sooner than you can deliver. • You need to know the minimum development time. Kolinger Associates, LLC 2010 All Rights Reserved
  34. 34. Crashing the Schedule Kolinger Associates, LLC 2010 All Rights Reserved
  35. 35. Effects of Schedule Compression - 1 What’s the effect of a 15% schedule reduction? Before Optimized Compressed Project Size: 100 KLOC Team Size: 11 Cost $1.5 MM Duration 23 months MTTD: 2.7 days Can we go faster with more people? QSM Kolinger Associates, LLC 2010 All Rights Reserved
  36. 36. Effects of Schedule Compression - 2 What’s the effect of a 15% schedule reduction? Before After Optimized Compressed Project Size: 100 KLOC Same Team Size: 11 24 Cost $1.5 MM $2.7 MM Duration 23 months 20 months MTTD: 2.7 days 1 day QSM Worth it? Really? Kolinger Associates, LLC 2010 All Rights Reserved
  37. 37. Effects of Schedule Compression - 3 Quality Considerations A B 20 Months 23 Months Defects / Mistakes Time • An individual’s learning curve must be accounted for in a shorter schedule. • There is a minimum development time! Kolinger Associates, LLC 2010 All Rights Reserved
  38. 38. Software Development Effort - 9 Projects Intense Schedule Pressure Schedule Cost Kolinger Associates, LLC 2010 All Rights Reserved
  39. 39. Deadline Effect Kolinger Associates, LLC 2010 All Rights Reserved
  40. 40. Improving Process Reduces Costs For a typical software system, consisting of 500K Lines of Code: SEI Effort Defects Total Cost Level (P. Mos.) (Number) (Mill. $) I 16,362 25,069 163.3 II 6,468 9,909 64.7 III 1,876 2,874 18.8 IV 866 1,326 8.7 V 342 524 3.4 Significant savings as a result of achieving Level 2 Most of these improvements come from better project management! Stan Rifkin, Master Systems Larry Putnam, QSM Ray Dion, Raytheon Kolinger Associates, LLC 2010 All Rights Reserved
  41. 41. Right-sizing ... or Dumb-sizing? Two Competitors, Same Products, Same Sales Revenues, Different Cultures Bureaucratic Robust Core Management Core Fixers Management 24% 29% 31% Process Fixers Improvement 63% 13% Process Improvement 40% Total Employees = 152 Total Employees = 80 Mark Blaxill, The Fallacy of the Quick Overhead Fix, July 1991 Harvard Business Review Kolinger Associates, LLC 2010 All Rights Reserved
  42. 42. Main Build – Estimated Duration Kolinger Associates, LLC 2010 All Rights Reserved
  43. 43. Main Build – Estimated Duration Kolinger Associates, LLC 2010 All Rights Reserved
  44. 44. Main Build Time - 9 Projects Historical Data Months Duration Industry Average ESLOC (Thousands) Kolinger Associates, LLC 2010 All Rights Reserved
  45. 45. Main Build Effort - 9 Projects Historical Data Person Months ESLOC (Thousands) Kolinger Associates, LLC 2010 All Rights Reserved
  46. 46. Project Staffing Comparisons 35.0 61% Company Peak Staff Projects from XYZ Division Benchmark Peak Staff 30.0 108% 25.0 Percent by which this project exceeds the typical 77% business systems average staffing 20.0 64% -11% Projects from ABC Division 15.0 0% 10.0 0% 180% -8.3% -25% 0% 5.0 0.0 A B C D E F G H I J K Kolinger Associates, LLC 2010 All Rights Reserved
  47. 47. A Business Case for Process Improvement at Company X 50 KLOC, Productivity Index (PI)=15, MBI=0.2 SEI Calendar Time Effort Defects Total Cost (1,000s) Level (Mos) (PMos) Discovered Shipped Median Lowest Highest I 18 67 297 11 614 202 20901 II 11 16 71 3 148 109 196 III 9 9 40 1 82 58 105 IV 7.5 5 22 1 44 32 57 V 5 2.5 11 0 23 5 31 Kolinger Associates, LLC 2010 All Rights Reserved
  48. 48. Benchmark Power One company could save $7 million in a portfolio of 14 projects by making the right improvements in their management approach. In addition, this portfolio of projects would finish 3.5 months sooner and with a much higher reliability/quality. This remarkable analysis is made possible by using the high-performance benchmark database and SLIM toolset. Kolinger Associates, LLC 2010 All Rights Reserved
  49. 49. Know your capability !!! Commitments must be tempered with known capability! Project Difficulty Management Capability Question: • For your current project, where would you look for a mismatch of commitment and capability? • How should you respond? Kolinger Associates, LLC 2010 All Rights Reserved
  50. 50. 6. When You Have to Micro-Manage People - Place them where strengths are leveraged, and their weaknesses really don’t matter. – People aren’t fungible – Consider: • 85% feel they are in the wrong job • 10% are in their dream job • 95% confess they lie on the job – Misplaced people will disrupt “It’s time for performance review!” – Fear repels learning and improvement – Focus on developing strengths not fixing weaknesses – Company slogans and performance reviews demoralize “Malicious Compliance” – be careful what you ask for! You might get it! Kolinger Associates, LLC 2010 All Rights Reserved
  51. 51. Kolinger Associates, LLC 2010 All Rights Reserved
  52. 52. 7. When You Hear Any Answer That Starts With, “Well …” Look for effective project reviews and the signs of being off-track • A ‘perfect’ estimate is valid as long as you manage to the plan. • Avoid use of ‘percent complete.’ – Remember the “90:90 Rule of Project Tracking.” • The first 90% of the project takes 90% of the work. • The last 10% takes the other 90%! • Do not be bamboozled in project reviews! Verify: – 1. Do we have the right people? – 2. Do we have a good plan? – 3. Are we going to make it? Kolinger Associates, LLC 2010 All Rights Reserved
  53. 53. “Relax – We’re 90 Percent Complete!” Traditional software progress measures do not provide visibility Software Engineering Economics, Boehm – pp 608 Kolinger Associates, LLC 2010 All Rights Reserved
  54. 54. 7 Signs – A Review 1. When You Hear the Word ‘Done’ 4 Times. So, what the heck does ‘done’ look like? 2. When You Hear, “What is the Plan?” Look for the plan with evidence of traceability to the project definition and make sure everyone is aware. 3. When You Have a Single Number Estimate Look for a schedule based on appropriate estimate ranges 4. When It Has Not Been Done Before Look for an estimate that factors in data of your organization’s capability 5. When You Can’t See Evidence of Historical Data Look for an estimate that uses historical performance data as a reference 6. When You Have to Micro-Manage Get the right people in the right place 7. When You Hear Any Answer That Starts With, “Well …” Look for effective project reviews and the signs of being off-track Kolinger Associates, LLC 2010 All Rights Reserved
  55. 55. Conclusion You must protect your company from a bad estimate • Radical improvement is possible • Information is not enough • Needed: 1. Invest in knowing your capability 2. Learn to make commitments according to capability 3. An outside perspective may help Kolinger Associates, LLC 2010 All Rights Reserved
  56. 56. Acknowledgements My thanks to the following individuals who have made important contributions to the body of knowledge presented in this document. • Gopal Kapur – Center for Project Management • Dr. Stan Rifkin – Master Systems • Dr. Robert N. Charette – ITABHI Corporation • Dr. Barry Boehm - USC Center for Software Engineering • Dr. Gerald Chester – Strategies@Work • Dr. Michael Deutch – Hughes Aerospace • Dr. Robert Graham - SMC • Tom DeMarco – Atlantic Systems Guild • Larry Putnam, Sr. – Quantitative Software Management • Dan Swaigen – PM Connect • Gordon Landies – GL Ventures • John Maher, CPA – Maher Accountancy • Dennis Peacocke – SCS • Alan Lashbrook – AT&T • Ron Linski – Alliance Business Consulting • Betsy Guthrie - Autodesk Kolinger Associates, LLC 2010 All Rights Reserved
  57. 57. About Us Kolinger Associates provides solutions and advice for better estimating and managing large projects. With our help you will … Spend your money a little differently … and get a much better result. • Process assessment and development • Planning and Estimating Training • People Assessment • Rescue Off-Track Projects • Tools Training and Implementation • Project Reviews Contact 415-246-7264 – joe@kolinger.net - www.kolinger.net Kolinger Associates, LLC 2010 All Rights Reserved

×