Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Software project failure in new zealand – what’s killing us and what we can do about it

1,319 views

Published on

Delivered at AgileNZ 2012 conference and ANZTB Conference Canberra, Australia June 2013

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

Software project failure in new zealand – what’s killing us and what we can do about it

  1. 1. Commercial in Confidence | Assurity Consulting Limited 2013Software project failure in NZwhat’s killing us and what we can do about itEdwin DandoConsulting Manager
  2. 2. Commercial in Confidence | Assurity Consulting Limited 2013Why study software project failure?• We all know software projects areoften challenged right ?• But just how challenged are they?• And are NZ projects any morechallenged than others?• And if so why?Curiosity got the better ofme….
  3. 3. Commercial in Confidence | Assurity Consulting Limited 2013Aim of the study• Provide up-to-date info about failure patterns in softwaredevelopment projects in NZ• Ali Baigy – my Persian Otago MBA friend• 16 weeks• 28 organizations including> 18 software companies> 3 universities
  4. 4. Commercial in Confidence | Assurity Consulting Limited 2013Why bother?1. Knowledge of software project failure in NZ is limited, out-of-date, incomplete, or irrelevant2. Our industry is in a crisis• People believe they can demand it therefore it can be done• Under pressure, developers cut quality to support this belief• The results include˗ Developers and customers resenting the profession˗ Failing products, failing companies, and hatefulworkWe are there now
  5. 5. Commercial in Confidence | Assurity Consulting Limited 2013
  6. 6. Commercial in Confidence | Assurity Consulting Limited 2013
  7. 7. Commercial in Confidence | Assurity Consulting Limited 2013
  8. 8. Commercial in Confidence | Assurity Consulting Limited 2013Approach• Review international studies> What are the common project failure modes from overseas?> Guide questions to see if we are the same• Approach people> Call first to consider a recent project in their org> Either send questionnaire or do over phone
  9. 9. Commercial in Confidence | Assurity Consulting Limited 2013The Participants
  10. 10. Commercial in Confidence | Assurity Consulting Limited 2013Main Location024681012Auckland Wellington Christchurch Dunedin
  11. 11. Commercial in Confidence | Assurity Consulting Limited 2013Industries/marketsBanks, insurance andfinancial services13%Government11%Education14%Energy7%Industry17%Product softwarecompanies5%Professional services(Accounting, Warehousing, Healthcare, etc)15%Retail8%Telecommunications8%Other2%
  12. 12. Commercial in Confidence | Assurity Consulting Limited 2013Number of FTEs01234567Less than 6 6 to 20 21 to 100 101 to 500 More than 500
  13. 13. Commercial in Confidence | Assurity Consulting Limited 2013Revenue0 1 2 3 4 5 6 7 8No answerLess than NZ$ 500KNZ$ 500K to NZ$ 2 millionNZ$ 2 million to NZ$ 10 millionNZ$ 10 million to NZ$ 100 millionMore than NZ$ 100 million
  14. 14. Commercial in Confidence | Assurity Consulting Limited 2013
  15. 15. Commercial in Confidence | Assurity Consulting Limited 2013Success rates0%10%20%30%40%50%60%Failure In Between Success
  16. 16. Commercial in Confidence | Assurity Consulting Limited 2013Types of Development0%10%20%30%40%50%60%70%80%Custom Software Development General Software Development A mixture of bothFailure In Between SuccessThe majority of the Custom Software Development(CSD) projects failed
  17. 17. Commercial in Confidence | Assurity Consulting Limited 2013Failure in their words…• “A failure because although the delivery was on time, and to budget, thebusiness did not take it up and failed as a consequence.”• “It earned our company a lot of money, but disrupted our staffing, wasunpleasant to work on, was not fully used by the client and was largelyirrelevant to the company as a whole.”• “From a company perspective it was a success; it has generated sales andpaved the way for us to enter new markets. From a developmentstandpoint it was a failure; it was significantly late and of lower qualitythan we expect.”
  18. 18. Commercial in Confidence | Assurity Consulting Limited 2013What is success?• From an IT provider:“Yes, [the project was a success] becausewe carried out the clients specs”• but when we dug deeper:“The project sponsordidnt like the end result.”
  19. 19. Commercial in Confidence | Assurity Consulting Limited 2013But this is often normal, right?
  20. 20. Commercial in Confidence | Assurity Consulting Limited 2013We carried out the clients specsBut hold on, we already know that• 35% of requirements change throughout project• 33-66% of requirements are seldom or never usedThe client doesn’t know what they want.Isnt just following their specs a bit irresponsible?
  21. 21. Commercial in Confidence | Assurity Consulting Limited 2013So what are people losing sleep over?
  22. 22. Commercial in Confidence | Assurity Consulting Limited 2013Major concerns0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%ConsultantDeveloper Team LeaderDevelopment ManagerGM OperationsIS ManagerManaging DirectorProject managerQuality ManagerSupport ManagerCost Quality Scope TimeSenior managers most worried about costSupport and QA managers mostworried about quality & scopeFor Consultant time&cost most importantFor Dev Lead time much moreimportant than qualityNo surprises here?
  23. 23. Commercial in Confidence | Assurity Consulting Limited 2013Challenges0%10%20%30%40%50%60%70%80%90%FailureIn BetweenSuccessOn failed projects, People, Communication and Scope = biggest challengesOn successful projects, Time, Cost, Estimation & Scope = biggest challenges
  24. 24. Commercial in Confidence | Assurity Consulting Limited 2013Business case?0%10%20%30%40%50%60%70%80%90%100%Failure In Between SuccessNoYes• Failed projects 27%• Successful 57%• In between 67%
  25. 25. Commercial in Confidence | Assurity Consulting Limited 2013Could the team specify software purpose & use?0%10%20%30%40%50%60%70%80%90%100%No Partly Yes No answerFailureIn BetweenSuccess• 100% on successful projects• 50% on failed projects
  26. 26. Commercial in Confidence | Assurity Consulting Limited 2013Amount of unmanaged scope change012345678Failure In Between SuccessUnmanaged scope change contributes significantly to failureNumberofunmanagedscopechanges
  27. 27. Commercial in Confidence | Assurity Consulting Limited 2013Over-runs0%20%40%60%80%100%120%Failure In Between SuccessOver cost Over scheduleFailed projects cost twice as much & take 110% longerthan planned
  28. 28. Commercial in Confidence | Assurity Consulting Limited 2013Planning0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5FailureIn BetweenSuccess• Importance of planning in all categories was moderate to high• A lack of planning was not a failure factor for the projects in this study
  29. 29. Commercial in Confidence | Assurity Consulting Limited 2013Importance of Leadership• “There was uncertainty over who was responsible for what, largely due toour manager not being able to give the project as much attention asrequired.”• “Leadership was part of the problem. A sponsor with unrealisticexpectations and a PM managing a number of projects and really justadministrating.”• “Inexperienced project management and lack of a true project sponsor didnot allow the outputs of the project to be worked through with the endusers.”• “The leadership is strong - but allows the individuals to contribute andchallenge the leadership and how the project is run. Discussion isencouraged.”
  30. 30. Commercial in Confidence | Assurity Consulting Limited 2013Importance of management support0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5FailureIn BetweenSuccessExtremely Low Low Neutral High Extremely highIncreased management support tends to result in higherchance of success
  31. 31. Commercial in Confidence | Assurity Consulting Limited 2013Development team expertise0 1 2 3 4 5FailureIn BetweenSuccessAbility to carry out tasks effectivelyAbility to understand humanimplications of a new systemAbility to work effectively as a teamAbility to work with top managementAbility to work with uncertainobjectivesExtremely Low Low Neutral High Extremely highMore experienced team tends to equal higher chance of success
  32. 32. Commercial in Confidence | Assurity Consulting Limited 2013Learning0%10%20%30%40%50%60%70%No, there was no process Yes, at the end of the project Yes, regularlyFailureIn BetweenSuccess• < 15% failed projects had regular learning process throughout the project.• Successful projects tended to have a learning process• Lack of a learning process can be considered a contributing factor.
  33. 33. Commercial in Confidence | Assurity Consulting Limited 201311 Key Findings1. Software projects fail in all types of organisations, regardlessof size, income, location, number of projects per annum2. 54% of projects in the study were failures and failed projectscost twice as much & take 110% longer than planned3. Custom software development is vastly (10 times) moreprone to failure than integrating existing software4. Concern areas> Senior managers – cost> Quality and Support managers – scope and quality> Development Managers – time> Consultants – time and cost
  34. 34. Commercial in Confidence | Assurity Consulting Limited 201311 Key Findings5. Biggest challenges> Failed projects - People, Communication and Scope> Successful projects - Time, Cost, Estimation and Scope6. Most failed projects don’t have a business case whereasmost successful ones do7. Scope clarity> On 100% of successful projects, the development team couldspecify the purpose & usage of the software> In less than 50% of failed projects the development team couldspecify the purpose & usage of the software
  35. 35. Commercial in Confidence | Assurity Consulting Limited 201311 Key Findings8. On failed projects scope changed significantly and notmanaging this scope change contributed significantly tofailure9. We dont tend to have good processes to learn from ourmistakes10. More planning doesn’t make any difference11. Leadership is critical
  36. 36. Commercial in Confidence | Assurity Consulting Limited 2013
  37. 37. Commercial in Confidence | Assurity Consulting Limited 2013The 5 Core Characteristics1. Majority (54%) fail, cost twice as much, take 110% longer anddon’t tend to have a business case2. Biggest challenges are People, Communication and Scope3. Scope> Dev team couldn’t specify purpose & usage of the software> Scope changes significantly> Poor processes for managing scope change> Planning more doesn’t help4. Custom dev = is 10 x more prone to failure than integrating5. Lack leadership and management buy in
  38. 38. Commercial in Confidence | Assurity Consulting Limited 2013Lets go deeper on that…
  39. 39. Commercial in Confidence | Assurity Consulting Limited 2013Sanity Check…• Cross reference against fresh NZ study> Software Development Practices in New Zealand, TheUniversity of Auckland Computer Science Department, Feb2012 - Diana Kirk and Ewan Tempero• And their findings?
  40. 40. Commercial in Confidence | Assurity Consulting Limited 2013She’ll be right Trev• “Issues with practices relating to quality”> ineffective or missing design-and-code-checkingpractices (e.g. reviews, unit tests)> a lack of independent testing> insufficient separation of environments> major issues with clarity and accessibility ofrequirements.“Represents a possible gap endemic in the NZapproach to developing software”
  41. 41. Commercial in Confidence | Assurity Consulting Limited 2013Process• “Organisations do not follow standardmodels such as Waterfall, Scrum orXP, but rather adapt practices”• “A majority of respondents claimed tobe ‘agile’, but close informal contactwith customers was not practiced”“Individuals do not follow practices in a consistentway. The risk of gaps is clearly large in the NewZealand context.”
  42. 42. Commercial in Confidence | Assurity Consulting Limited 2013So lets make that 7 CoreCharacteristics…1. Majority (54%) fail, cost twice as much, take 110% longer and don’t tend tohave a business case2. Biggest challenges are People, Communication and Scope3. Scope> Dev team couldn’t specify purpose & usage of the software> Scope changes significantly> Poor processes for managing scope change> Planning more doesn’t make any difference4. Custom dev = is 10 x more prone to failure than integrating5. Lack leadership and management buy in6. Issues with practices relating to quality7. Organisations do not follow standard models but rather adapt practices
  43. 43. Commercial in Confidence | Assurity Consulting Limited 2013Back to the investigation…
  44. 44. Commercial in Confidence | Assurity Consulting Limited 2013#1 - Majority (54%) fail, cost twice as much, take110% longer & don’t have a business case• Consistent with Product Ownership “Achilles heel”> Why are we doing the project?> What value does each feature add?> Should we continue this project?> How can we trim the tail to get an early working version into customer hands?• Consistent with my own observations• One solution – quality Product Ownership> Work with customers + people developing software> Work based on value> Validate assumptions and reduce risk via regular feedback cycles
  45. 45. Commercial in Confidence | Assurity Consulting Limited 2013# 2 People, Communications & Scope
  46. 46. Commercial in Confidence | Assurity Consulting Limited 2013# 2 People, Communication and Scope• But done properly agile nailspeople, communication and scope!• Oh, that’s right: “organisations do notfollow standard models but rather adaptpractices”• Epic Fail
  47. 47. Commercial in Confidence | Assurity Consulting Limited 2013# 3 - Scope• Agile gives you> Ability to change scope frequently> Stronger people focus through self-management and empowerment> Increased communicationsAll of this for free!!!
  48. 48. Commercial in Confidence | Assurity Consulting Limited 2013The Sprint Agreement
  49. 49. Commercial in Confidence | Assurity Consulting Limited 2013# 4 custom dev = is 10 x more prone to failurethan integrating• But this is where the magic happens!• Innovative products aren’t the result ofintegration• Did Steve Jobs integrate existing products?• Did Yikebike integrate existing products?• Did Lars & Jens Rasmussen integrate – no theybuilt!• We have a system that fosters innovation whilemanaging complexity and controlling risk….
  50. 50. Commercial in Confidence | Assurity Consulting Limited 2013# 5 Leadership & Management Buy In• Servant Leadership – behaviours are values in action• Predictive management versus empirical management• The desire to appear certain vs. the courage to admit software isnot predictable and risk is best managed incrementally, with data• Authority moves down the organization – trust your people!NZ/AU - Strengths toOpportunities• Use ourisolation, innovation, attitude and independence tolead.• Harness our pioneeringinnovation
  51. 51. Commercial in Confidence | Assurity Consulting Limited 2013The changing role of management• People dont set out to fail each day - they set out to succeed• Successful managers> create an environment for success> empower their people> stand aside• Define the "what" not the "how"• Accept that people dont work for money, they work for AMP> Autonomy, Mastery, Purpose• The world is changing and Gen-Y are helping challenge the statusquo• Manage them as you would a team of volunteers
  52. 52. Commercial in Confidence | Assurity Consulting Limited 2013# 6 - Issues with practices relating toquality• missing design-and-code-checking practices• a lack of independent testing• insufficient separation of environments• major issues with clarity and accessibility of requirements.• Agile will highlight these insufficiencies quicklyand often (like your mother in law coming tostay – you will be reminded of your deficienciesevery day….)“Scrum will highlight every deficiency andimpediment that the enterprise has so theenterprise can fix them and change into thebest organization in its market”Ken Schwaber, Scrum is Hard & Disruptive, 2006
  53. 53. Commercial in Confidence | Assurity Consulting Limited 2013• The practices exist for good reason. Don’t skip themuntil you have mastered the basics“Scrum is not a methodology that needs enhancing. That is how we got into trouble inthe first place, thinking that the problem was not having a perfect methodology. Effortcentres on the changes in the enterprise that are needed”- Ken Schwaber, Scrum is Hard & Disruptive, 2006The student follows theteachings precisely.Concentrates on thetask, without worryingabout theory.Basic practices working- student starts tounderstand theunderlying principles &branch out.Student isnt learning fromother people, rather ownpractice. Creates ownapproaches & adaptslearning to circumstances# 7 - Organisations don’t follow standardmodels - they adapt practices
  54. 54. Commercial in Confidence | Assurity Consulting Limited 2013Finally - Complexity• The basic problem lies in ourcomprehension of whatcomplexity is.• Paradoxically, the deeper we lookinto complexity the more we see.• As a result, we now appreciatethat prediction of complexsystems is (probably) impossibleNewcastle UniversityEngineering Design Centre
  55. 55. Commercial in Confidence | Assurity Consulting Limited 2013Contrasting Project FailureFailure, 54%InBetween,22%Success,26%New Zealand• We succeed less• We fail more• We have less in betweenSuccessFailureChallengedSuccessFailureChallenged
  56. 56. Commercial in Confidence | Assurity Consulting Limited 2013SummaryWe are failing for silly reasons and we know better• Product Ownership - business case, communication, clear view of productpurpose & usage, ability to change and manage scope• People deliver projects – team need a clear view of the product purpose &usage, plus the ability to get involved in this together• Leadership is critical – you are only a leader if people follow• Values trump everything – an open “safe to fail” culture wins• Journey of constant improvement - effort centres on the changes in theenterprise• Start with the practices and only adapt once you know what you are doing• Don’t be scared of custom dev – just actively manage the risk usingtransparency and empirical processes
  57. 57. Commercial in Confidence | Assurity Consulting Limited 2013The future…• Failure isnt bad, as long as we learn from it• Our industry is a wonderful industry full of amazing peoplewho want to succeed• But we must be brave, creative and honest – past managementthinking is failing us• We can use isolation to reduce negative influences to followand lead in our own way
  58. 58. Commercial in Confidence | Assurity Consulting Limited 2013Questions?

×