Your SlideShare is downloading. ×
Early Warning Signs of IT Project Failure -- The Deadly Dozen and the Four Horsemen of Doom
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Early Warning Signs of IT Project Failure -- The Deadly Dozen and the Four Horsemen of Doom


Published on

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. The Early Warning Signs y g g of IT Project Failure: The Deadly Dozen & The F Th Four Horsemen of Doom H fD Leon A. K L A Kappelman, Ph.D. l Ph D Professor of Information Systems y Director Emeritus, IS Research Center Fellow, Texas Center for Digital Knowledge Information Technology & Decision Sciences Dept. College of Business, University of North Texas Business Founding Chair, SIM Enterprise Architecture Working Group ( Website: Email: Phone: 940-565-4698 Fax: 940-565-4935 pp@ © 2010 Leon A. Kappelman Baton Rouge: 5-Oct-2010
  • 2. No Shortage of Major IT Project Failures • FoxMeyer Drugs ** • McDonalds • Denver Airport Baggage p gg g • Shane ** • Internal Revenue Service • KMart ** • Dept. of Veterans Affairs • Nike • London Stock Exchange d S kE h • AMR • Bank of America • FAA • Hershey Foods • FBI • American LaFrance ** • IRS FoxMeyer Drugs – a $5 billion annual revenue drug distributor. First billion dollar bankruptcy due to a failed IT project. ** = IT directly implicated in bankruptcy. 2
  • 3. The Continuing IT Project Challenge 19% canceled - “outright failure” 46% “cost or time overruns … didn’t cost didn t Canceled 19% fully meet user’s needs” Late, Over Only 35% completed on time, on Budget, budget, ith b d t with promised functionality i df ti lit Reduced Scope Initial performance & reliability are 46% Successful S f l often less than expected & needed p 35% Cancellation rate increases w/size: – 32% large projects cancelled (> 10K FPs) – 52% very large (> 100K FPs) FPs = Function Points Capers Jones, 2009 2006 Chaos Report, The Standish Group, 3
  • 4. Why Focus on Early Warning Signs? • The cost of fixing an error rises dramatically over time. Requirements – “No other part of the work so cripples the system if done wrong. No other part is more difficult to rectify later.” – Fred Brooks, No S Silver Bullets (1986) • Human Nature • IT people are optimists l ti i t • Escalation Theory • Admitting wrong/don t know wrong/don’t • Sunk cost • Job security • Recency bias 4
  • 5. What are the sources of IT project risk?
  • 6. Three Main Sources of IT Project Risk Th M i S f P j t Ri k •People p •Process •Product 6
  • 7. Requirements Ain’t Easy! Perspectives of People Vary – a lot Cook, M. (Speaker). (26 February 2005). Scorecards and Behavior Checklists as a Method of Measuring Process Deployment Across the Organization [presentation] Plano, TX: SEI Software Engineering Process Improvement Workshop EDS Auditorium [presentation]. Plano Workshop, Auditorium. Requirements change at an average rate of 1.6% per month (1K to 10K FPs), or about 34% over the life of a 10,000 FP project (Capers Jones, 2008) 7
  • 8. PROCESS IS CRITICAL MILESTONES & INSPECTIONS MATTER! When you’ve completed a step and are ready to progress into your next step, you must hold a Project Decision Meeting and get the MDA’s approval to continue. 8
  • 9. Project Size Matters! j Probability of Selected IT Project Outcomes Source: Capers Jones (1996) 9
  • 10. Source: Wallace, Keil, & Rai (2004) 10
  • 11. PEOPLE PRODUCT Source: Wallace, Keil, & Rai (2004) 11
  • 12. 13 © 2001-10 Leon A. Kappelman
  • 13. Best Practices are Known Retrospectives conducted in 74 organizations over the prior sever se er (7) years ears Source: R. Ryan Nelson (2007) 14
  • 14. Copyright © 2008-2009 by Capers Jones & Associates LLC All rights reserved.
  • 15. Tools Can Help RESULTS >10,000 FUNCTION POINT PROJECTS Probability of Selected Outcomes Cancel Delays On time Early Automated estimates 1% 2% 78% 19% Automated plans Formal tracking Optimal quality control Manual estimates 40% 45% 15% 0% Manual plans Informal tracking Minimal quality control Source: Capers Jones (1996)
  • 16. EWS Research Goal Identify relative importance of Early y p y Warning Signs (EWS) of impending IT Project Failure within the first 20% of the original project schedule … While there is still time to get back on track to success at a reasonable cost cost. IT Project Success is defined as being completed on-time, on-budget, with the originally promised functionality and necessary performance and reliability, all adjusted accordingly for changes in requirements requirements. 18
  • 17. Our Research Approach: Three Phases 1. 1 Extensive literature search 2. Panel of 15 Experts - Delphi Study 3. Survey of senior IT project managers – Average 15 years experience – Maximum project size of $3 million to $7 billion 19
  • 18. Survey • 53 possible Early Warning Sign factors • People, Process, and Technology EWS P l P dT h l • Developed an on-line survey tool p y • Rank each EWS factor from 1 to 7 1 – Extremely Unimportant 7 – Extremely Important 20
  • 19. 21 © 2001-10 Leon A. Kappelman
  • 20. EWS Research Findings g • 17 (out of 53) with a mean score > 6 (out of 7) • Th These 17 consolidated down to a Top 12 – lid t d d t T The “Deadly Dozen” Early Warning Signs • N T h i l factors made the top 40 No Technical f t d th t • Technical Factors were all in the bottom 20% IT projects almost never fail because of technical causes, despite the fact that people and process problems may manifest technically. The technical ailments of IT projects can be traced to people and process causes that exploit inherent product risks, such as large size, high complexity, or novel technology. Nevertheless, these technical , g p y, gy , risks can be mitigated with proper people and process practices. Even risks of large size projects, which are technical (product) risks, can b mitigated by good process and people. be iti t d b d d l
  • 21. Early Warning Signs “Deadly Dozen” People Factors Center on five not altogether mutually exclusive groups of people: Top Management, Project Management, Project Team Members, Subject M tt E S bj t Matter Experts (SME = users), & St k h ld t (SMEs ) Stakeholders i general. in l 1. 1 Lack of Top Management Support 2. Weak Project Manager 3. 3 No Stakeholder involvement and/or participation 4. Weak Commitment of Project Team members 5. 5 Team Members lack requisite knowledge or skills T M b l k i it k l d kill 6. Subject Matter Experts are over-scheduled
  • 22. Early Warning Signs “Deadly Dozen” Process Factors Center on five project management processes and their associated deliverables that are essential to success: Requirements (including B i (i l di a Business Case), Ch C ) Change C t l S h d l Control, Control, Schedule C t l Communications, and Resources. 1. Lack of Documented Requirements / Success Criteria 2. No Change Control Process 3. Ineffective Schedule Planning / Management 4. Communications Breakdown among Stakeholders 5. Resources assigned to a higher priority project 6. No Business Case for the Project
  • 23. Table 1: The Early Warning Signs of IT Project Failure The Four Horseman of IT Project Doom The Deadly Dozen EWSs Stakeholders Requirements Processes Team People-Related Risks 1. Lack of top management support. X 2. W k 2 Weak project manager. j X 3. No stakeholder involvement. X 4. Weak commitment of project team. X 5. 5 Team members lack req isite kno ledge and/or requisite knowledge X skills. 6. Subject matter experts overscheduled. X Process-Related Risks 7. Lack of documented requirements and/or success X criteria. 8. No change control process or change X management. 9. Ineffective schedule planning and/or X management. 10. Communication breakdown among X stakeholders. 11. R 11 Resources assigned t higher priority project. i d to hi h i it j t X 12. No business case for the project. X “Material Financial Risks of IT Projects: The Early Warning Signs of Failure” by Leon A Kappelman A. Kappelman, forthcoming in 2010 in the Interpreter a quarterly publication of the Insurance Accounting & Systems Association
  • 24. Fred Brooks on the difficulties of software development development… “The hardest single p of building a g part g To see what rate of progress one can expect in software system is deciding precisely software technology, let us examine the what to build gy, other part of the build. No difficulties of that technology. Following conceptual work is as difficult as Aristotle, Aristotle I divide the detailed technical establishing them into essence the essence, difficulties inherent in No other part software, requirements…. the nature of of the and work so cripples the system if doned d accidents, th id t those diffi lti that today difficulties th t t attend itsg No other but are more difficult to wrong. production p is not inherent. part rectify later.” "No Silver Bullet - Essence & Accidents of Software Engineering” No Engineering 1986 in Information Processing 86. H.J. Kugler, ed., Elsevier, 1069-1076. (Invited paper, IFIP Congress '86, Dublin) Reprinted in The Mythical Man-Month, 20th Anniversary Edition, Frederick P. Brooks, Jr., Addison-Wesley, 1995.
  • 25. Requirements = Architecture q Architecture? What’s that? • Architecture is “the set of descriptive p representations about an object.” [John Zachman] • Enterprise Architecture is “the holistic set of descriptions about the enterprise p p over time.“ [SIMEAWG] • Enterprise Architecture is modeling the enterprise. [LAK]
  • 26. Why do we need architectural models? Failure to plan is a plan for failure
  • 27. INDUSTRY DATA ON DEFECT ORIGINS Because defect removal is such a major cost element, studying defect origins is a valuable undertaking. IBM Corporation (MVS) SPR Corporation (client studies) 45% Design errors g 20% Requirements errors q 25% Coding errors 30% Design errors 20% Bad fixes 35% Coding errors 5% Documentation errors 10% Bad fixes 5% Administrative errors 5% Documentation errors 100% 100% TRW Corporation Mitre Corporation Nippon Electric Corp. 60% Design errors g 64% Design errors g 60% Design errors g 40% Coding errors 36% Coding errors 40% Coding errors 100% 100% 100% Copyright © 2009 by Capers Jones. All Rights Reserved. SWQUAL0839
  • 28. U.S. U S AVERAGES FOR SOFTWARE QUALITY (Data expressed in terms of defects per function point) Defect Removal Delivered Defect Origins Potential Efficiency Defects Requirements R i t 45% 1 00 1.00 77% 0.23 0 23 56% Design 1.25 85% 0.19 Coding 1.75 95% 0.09 Documents D t 0.60 0 60 80% 0.12 0 12 Bad Fixes 0.40 70% 0.12 TOTAL 5.00 5 00 85% 0.75 0 75 (Function points show all defect sources - not just coding defects) Copyright © 2009 by Capers Jones. All Rights Reserved. SWQUAL0840 © 2001-10 Leon A. Kappelman 40
  • 29. BEST IN CLASS SOFTWARE QUALITY ( (Data expressed in terms of defects p function p p per point) ) Defect Removal Delivered Defect Origins g Potential Efficiency y Defects Requirements 0.40 85% 0.08 40% 77% Design D i 0.60 0 60 97% 0.02 0 02 Coding 1.00 99% 0.01 Documents 0.40 98% 0.01 Bad Fixes 0.10 0 10 95% 0.01 0 01 TOTAL 2.50 96% 0.13 50% 17% OBSERVATIONS Most often found in systems software > SEI CMM Level 3 Copyright © 2009 by Capers Jones. All Rights Reserved. SWQUAL0841 © 2001-10 Leon A. Kappelman 41
  • 30. “No other part of the No SIMEAWG IT Management Practices difficult conceptual work is as Study Averages (Scale: 1[=awful] to 5 [=superior]) as establishing the detailed • 3.67 Overall average (64 q g ( questions) ) technical t h i l requirements…. N i t No • 3.92 Purpose / function of EA (7 questions) • 3.90other part of the work so Potential benefits of EA (20 questions) • 3.68cripples the system if done pp y ISD CMM practices and capabilities (12 questions) • 3.53 Use of requirements artifacts (10 questions) • 3.33wrong. No other part is(15 questions) Requirements practices & capabilities more difficult to rectify later.” – Fred The SIM Guide to Enterprise Architecture: Creating the Information Age Enterprise, 2010, edited by Leon A. Brooks Kappelman, CRC P K l Press, T l and F NYC, ( Taylor d Francis G i Group,
  • 31. The SIM Guide to Enterprise Architecture Creating the Information Age Enterprise 542KA = 40% off A project of the SIMEAWG Edited by: Leon A. Kappelman, Ph.D. Foreword by: Jeanne W. Ross, Ph.D. Contributing Authors, P C t ib ti A th Panelists, & Artists (alphabetically): li t A ti t ( l h b ti ll ) • Bruce V. Ballengee • George S. Paras • Larry Burgess • Alex Pettit • Ed Cannon • Jeanne W. Ross W • Larry R. DeBoever • Brian Salmans • Russell Douglas • Anna Sidorova • Randolph C. Hite C • Gary F Simons F. • Leon A. Kappelman • Kathie Sowell • Mark Lane • Tim Westbrock • Thomas McGinnis o as cG s • John A. Zachman Jo ac a All authors’ royalties support the work of the not-for-profit SIMEAWG.
  • 32. The act of discovery consists not in finding new lands fi di l d but in seeing with new eyes. y – Marcel Proust
  • 33. EWS of IT Project Failure Four Horseman of Doom •Requirements • Lack of documented requirements and/or success criteria q • Resources assigned to a higher priority project • No business case for the project •Stakeholders • Lack of top management support • No stakeholder involvement and/or participation • Subject matter experts are overscheduled •Project Team • Weak project manager • Weak commitment of project team • Team members lack requisite knowledge and/or skills •Project Management Processes • No change control process or change management • Communication breakdown among stakeholders • Ineffective schedule planning and/or management
  • 34. There is St No S e Bullet e e s Still o Silver u et The warning signs are clear -- if you are willing to look. But it is very hard for project stakeholders and participants to be objective. Hope i not a strategy, but that is t often th strategy. H is t t t b t th t i too ft the t t Success requires objectivity, candor, hard work, and hard h i h d choices.
  • 35. EWS to Lower Personal Career Risk • All participants are often damaged. p p g • Project recovery miracles are very rare. • A project “death march” rarely succeeds. death march succeeds • Most CFOs have been burned enough that credibility of project status reports are low. dibilit f j t t t t l • Big IT projects are very visible (material risk). • Outsourcing more (even all) of IT can result. • Career damage can result. g • Use EWSs awareness to be the winner who is credible and delivers!! Improve your career prospects. 47
  • 36. What can you do? y • Pay attention! – See with “new eyes” • Make all stakeholders aware of EWS risks from the beginning – feasibility, planning, funding. • Outside experts can be helpful & objective – When it’s tough for stakeholders to be objective. – When knowledge transfer desired. • If EWSs are identified, then ask: – How can the EWS be fixed, managed, mitigated? – Is the business case still valid? – Is stopping the project the best answer? • S k costs are sunk – th Sunk t k throwing good money after bad is foolish. i d ft b d i f li h • Human behavior is sometimes obtuse – Escalation Theory (and market bubbles) – but you don’t have to be “stupid on purpose”. • Candor + Courage = Credibility
  • 37. “No one has to change No change. Survival is optional.” optional – Dr. W. Edwards Deming
  • 38. The Early Warning Signs of IT Project Failure: The Deadly Dozen & The F Th Four Horsemen of Doom H fD Leon A. K L A Kappelman, Ph.D. l Ph D Professor of Information Systems y Director Emeritus, IS Research Center Fellow, Texas Center for Digital Knowledge Information Technology & Decision Sciences Dept. College of Business, University of North Texas Business Founding Chair, SIM Enterprise Architecture Working Group ( Website: Email: Phone: 940-565-4698 Fax: 940-565-4935 pp@ © 2010 Leon A. Kappelman Baton Rouge: 5-Oct-2010