Presentation by IITP CEO Paul Matthews to the 2014 ALGIM (Association of Local Govt Information Managers) conference in Auckland, New Zealand.
The presentation looks at research into why software projects fail, the implications for procurement, and briefly outlines a different approach to procurement that IITP contends would be more compatible with software.
6. IT Project success and failure
42%
21%
37%
Major
cost/time/functionality
issues
Canceled before
completion
Success
Source: The Chaos Report 2012
7. Project success over time
0%
10%
20%
30%
40%
50%
60%
1994 1996 1998 2000 2004 2006 2008 2010 2012
Successful
Failed
Challenged
Source: The Chaos Reports and Manfesto 1995-2013
8. Other software project stats:
McKinsey (2012)
– Average cost overrun: 66%
– Average time overrun: 33%
– Limited functionality: 17%
Gartner (2012)
– Project failure: 20-28%
– Heavily reliant on project size
McKinsey. (2012). Delivering large-scale IT projects on time, on budget, and on value.
Gartner. (2012). Survey Shows Why Projects Fail. In L. Mieritz (Ed.): Gartner.
9. Fact 1:
Too many software projects fail.
Few would argue that something
isn’t wrong – software projects are
failing at an alarming rate.
16. How often is price estimation right?
0%
10%
20%
30%
40%
50%
60%
70%
Never Sometimes Usually Always
Small
Large
Source: IITP Software and Procurement Survey (2014)
21. Project success over time
0%
10%
20%
30%
40%
50%
60%
1994 1996 1998 2000 2004 2006 2008 2010 2012
Successful
Failed
Challenged
Source: The Chaos Reports and Manfesto 1995-2013
22. Project success over time
0%
10%
20%
30%
40%
50%
60%
1994 1996 1998 2000 2004 2006 2008 2010 2012
Successful
Failed
Challenged
Source: The Chaos Reports and Manfesto 1995-2013
2% of all
5% of new
9% of all
22% of new
24. Agile and NZ Procurement Compatibility
0%
10%
20%
30%
40%
50%
60%
70%
80%
Not compatible Reasonably Very
Source: Those with Agile experience from IITP Software and Procurement Survey (2014)
25. Fact 4:
Agile works better for software.
Agile succeeds 3x as often as Waterfall,
but isn’t very compatible with how we
[currently] procure software.
27. Too many software projects fail.
Big, complex projects usually fail.
We generally can’t predict the cost of
[complex] software up front.
Agile works better for software.
1
2
4
3
29. Agile and NZ Procurement Compatibility
0%
10%
20%
30%
40%
50%
60%
70%
80%
Not compatible Reasonably Very
Source: Those with Agile experience from IITP Software and Procurement Survey (2014)
36. NZ Roading Projects
Low Risk
Risk and Innovation
High Risk and Complex
Traditional Procurement
Alliance Model (= QBS)
Design and Build
Risk and
Complexity
37. Fact 5:
QBS is a credible alternative
Qualifications-Based Selection is
proven successful in high-complexity
scenarios, and deals with the “big 5”.
38. QBS in the Alliance Model
has a 100% success rate in
the NZ roading sector
Procurement – eyes glaze over.
Sorry to Procurement folks, but this is NOT a presentation about procurement. Well, mostly.
In fact, it’s a talk about software projects and more importantly, why they fail.
So the first half of this presentation is going to be about facts and figures – lots of graphs and numbers
Then we’ll talk about what this means in the context of procurement, and also about some of the industries that have had this problem before us and what they did about it
*** 5 FACTS
So do software projects fail more than other projects, and how bad is the problem?
Then “characteristics” of successful and failed software projects
So if we define the characteristics of project failure, how compatible are they with price-based procurement?
And from a procurement sense, are there lessons we can learn from other sectors – is there a way that’s more compatible with the point in 2
DEFINE!!!
Chaos Report:
Successful: Projects that don’t experience significant cost overruns, time overruns or loss of significant functionality;
Failed: Projects that are discontinued (cancelled) before they are successfully implemented;
Challenged: Projects that experience significant cost or time overruns or are implemented with significantly less functionality than intended.
Note on Chaos Report
Hazard a guess
Can you imagine if this was a roading project?
Is improving over time – some good solid reasons for that.
A couple of different contexts
Anyone want to hazard a guess at the proportion of tech projects fail because of technology (vs people)?
Software projects are inherently difficult to price up-front
Constructive Cost Model (COCOMO)
So if we take away small projects for a minute, given we know the biggest issue is with large ones
Again, large projects far less likely to “take it on the chin”
What is Agile?
Proposed: 2001, 12 core principles “Agile Manifesto”
Acceptance of rapid change
Small self-selecting teams and iterative development
Focus on rapid prototyping
Frequent delivery of working software
Working software over and above any other measurable outcome
Not: 69%
Very: 4%
So, why not?
Generalisation: Procurement focuses on defining a project up front, pricing it, then delivering to spec.
Software simply doesn’t work that way.
**IRONY** - larger the project, more process
New Zealand mandates a cost-based approach, generally meaning cost-up-front (although there are some exceptions).
Cost isn’t usually 100% of the rationale, but as with the Unions and the Labour Party, even 20% can be the deciding factor.
And…
So we have to ask, is there a better way?
So let’s change gears a little and talk about a thing called QBS.
Basic concept:
Purely a “qualifications-based” approach. Find the partner, not the solution.
No price discussions (specifically banned). Create an ordered list by competence for job.
Pricing *terms* are negotiated with top place in list, rather than price.
Both parties share in success or failure
Bridge
Two examples of usage.
US Federal Govt for “traditional” Engineering projects – since 1972. But not for software due to “IBM Effect”
NZ Transport Authority for roading projects