Software Process Improvement (SPI) Cost and ROI Ikram Ahmed Khan Business Beam (Pvt.) Limited [email_address]
An Eye Opening Experience!
One of the software house (lets name it XYZ) in Karachi bid for a project for a US based company.
Its competitor was a CMMI Level 5 company (lets name it CL5) from our neighbor.
Both competitors had no previous experience with the intended project domain.
XYZ bid with very low profit margin, aiming to “snatch” project from CL5.
XYZ came to know that CL5 bid less than half, both in time and cost, and won the project. ( Later: CL5 successfully delivered the project also )
Quotes by Some Software Industry “Gurus”
“ Why should I pay to testers? My client pays for development, not for testing!”
“ We implemented ISO9000:2000, but it proved to be a HUGE overhead.”
“ With tight schedules and even tighter budgets, we just can’t afford software process improvement.”
“ Efficiency is secondary to me. My first priority is code development!”
More Quotes by Some Software Industry “Gurus”
“ Software Process is just BS!!! I tried to follow process by book and it turned out to be a disaster. I ultimately had to do programming by myself.”
“ My team is not responsible at all. If I am with the project, it is a success, otherwise it’s a failure!”
Hallmarks of Our Industry
Its an unfortunate truth that our most projects are:
Late: deadlines are usually missed
Over budget: cost more than planned
Not fully functional: don’t have all the promised features and functionalities
Defective: deployed with defects and need heavy re-work to remove them.
We do the same thing over and over again, and expect different results.
Do you still feel that we should not spend on improvement?
Common (Mis)conceptions about Software Process
Investment in Process = More Cost
Industry operate at such a low margin that it cannot afford processes.
Processes increase overheads, resulting into more development and/or deployment time.
Processes makes an organization inefficient. Our time is consumed by documentation, rather than coding.
Realities about Software Process
Industry operates on low profits because it does not have efficient processes.
Engineers do lot of re-work because of incapability of existing processes.
If a process makes you do unnecessary work, then its not an efficient process.
An effective process must reduce your response time, increase productivity and stabilize your internal systems.
Questions about SPI
Is software process improvement worth the investment?
Can a mature process really pay for itself?
Is SEI/CMMI or Six Sigma goals worth pursuing?
What Improved Maturity can do for your Organization? Based on data from 1300 applications, average 200,000 SLOC Source: Master Systems, Inc., Data obtained from QSM Database, QSM, Inc. Presented by: Rodger C. Blair, Pittsburgh PMI Chapter, April 12, 2001 0.93M $0.52$ $0.73M 80 person months 15 7 3 $1.7M $0.96M $1.3M 143 person months 18.5 12 2 $100+M $1.8M $5.5M 600 person months 30 61 1 Highest Cost Lowest Cost Median Cost Level of Efforts Calendar Months Defects Shipped CMM Level
What Improved Maturity can do for your Organization? (2) * Cost per unit is measured in dollars (median cost) per source line of code ($ per SLOC) produced. ** Programmer productivity is measured in number lines of code produced per programmer per month (SLOC per programmer per month). 3.5 defects every 1000 SLOC (1.00) 2500 (7.5) $3.75 (1.00) 3 7.5 defects every 1000 SLOC (2.14) 1399 (4.2) $6.5 (1.73) 2 30 defects every 1000 SLOC (8.58) 333 (1.00) $27.5 (7.3) 1 Average Defect Density Average Programmer’s Productivity ** Average Unit Cost * CMM Level
Case Study 1: Raytheon
Business Need: Raytheon concluded “their processes
weren’t adequate to deal with the
diverse needs of customers.”
Proposed Solution: CMM Level 3
Source: Robert .B. Grady, Successful Software Process Improvement, Prentice-Hall, 1997.
Case Study 1: Raytheon Savings due to CMM level 3 SPI Efforts! | | | | | | | | | | | | | | | | | | | JAN MAY SEP JAN MAY SEP JAN MAY SEP JAN MAY SEP JAN MAY SEP JAN MAY SEP 1987 1988 1989 1990 1991 1992 0 5 10 15 20 25 30 35 40 45 50 Percentage of Total Project Cost Cost of Rework $15.8 Million Saved on $2.05M SPI Investment Start of SPI Initiative 41% 11%
Case Study 1: Raytheon (Cont..)
Raytheon SPI Results show:
7.7 to 1 ROI; achieved SEI/CMM Level 3.
Two-fold increase in productivity.
Cost of rework reduced from 41% to 11% of total project cost.
Eliminated $15.8M in rework due to reducing error-rate at a $1,281 SPI cost per engineer per year.
Case Study 2: SAIC Health Care Technology
Business Need: SAIC recognized ”that to remain
competitive and retain its customers, it
needed to develop products faster and
improve the quality of the delivered
Proposed Solution: CMM Level 2
Source: J. Lane and D. Zubrow, “Integrating Measurement with Improvement”, Proc. Of 19th IEEE International Conference on Software Engineering, May, 1997.
Case Study 2: SAIC CMM Level 2 SPI Results 20% 40% 60% 80% 100% V4.1 V4.2 Established Baseline Achieved CMM Level 2 Percentage Improvement x Cost dropped 30% x Cycle-time dropped 46% B x Error-rate dropped 90%
Objectives of Software Process Improvement Efforts
Each defects costs
Higher number of defects results into very high rework cost (and unsatisfied clients also)
What is the productivity of your developer? Do you have statistics?
The efficiency of development team depends on the process bottlenecks
Do you know the accurate project status at any point of development?
SPI Example 1: Software Process without Inspections Requirements Design Code Unit Test Integration Testing Systems / Acceptance Test Rework Rework 100 + 100 + 100 + Production Rework Rework Rework Rework Rework Rework 100 ++
SPI Example 1: Software Process with Inspections Requirements Inspection Rework Design Inspection Rework Code Inspection Rework Unit Test Integration Testing Systems / Acceptance Test Requirements Design Code 1 1 1 Production
Cost of SPI Efforts
For US organizations, the average cost per year per Software Engineer is $1,700 or within 5% range with 300 Software Professionals in 2001. *
Rodger C. Blair, Pittsburgh PMI Chapter, April 12, 2001
R. Dion, “Process Improvement and the Corporate Balance Sheet”, IEEE Software, Sept. 1994, pp. 23-30
A. Carleton, et al ., Benefits of CMM-Based Software Process Improvement: Initial Results, Tech. Report CMU/SEI-TR-94-013, Carnegie Mellon University, 1994
W. Lipke and K. Butler, “Software Process Improvement: A Success Story”, Crosstalk, Nov. 1992, pp. 29-39.
Cost of SPI Efforts (cont…)
For Pakistani organizations, the average cost per year per Software Engineer can be as follows:
2k – 3k 50-100 3k – 4k 25-50 4k – 5k 10-25 SPI Cost per Software Professionals per month (Rs.) Organization Size (Number of Software Professionals)
Cost of SPI Efforts (cont…) 2 4 6 8 10 12 14 16 18 20 Cost of Defect Removal (Rs. in 000) Defect Removal Stage Req. Analysis Design Code Test Acceptance Operations Code Defects Design Defects Analysis Defects Req. Defects
Cost of SPI Efforts (cont…)
Cost of internal defects may be between Rs.500 to Rs.10,000
Cost of post-deployment defects may be between Rs.5,000 to Rs.100,000+
How many defects (bugs) you had in your last project? You can calculate ROI on SPI Cost.
ROI of SPI efforts usually range from 2:1 to 9:1
How to have Successful SPI Program
Get services of a good SPI coach. Don’t follow do it yourself approach unless you have the required expertise.
Confirm executive level (CxO) sponsorship and commitment.
Treat SPI as a project. Develop plan, outline key success factors and provide resources.
Common Reasons for the Failure of SPI Efforts
Lack of executive-level (CxO) sponsorship and commitment.
Lack of middle and 1st-line management commitment.
Organizational culture unwilling to change -- e.g., “Hero-developer” culture.
Lack of commitment to continuous SPI improvement.
Do it yourself approach.
Continue with the software industry’s historical record.
Transform your company into an efficient and mature organization with higher profits
Thank you! You may find this presentation at: www.businessbeam.com