HCLT Research Paper: Cost of Quality


Published on

http://www.hcltech.com/enterprise-transformation-services/overview~ More on ETS

The purpose of this paper is to illustrate the significance of incorporating CoQ principles in software solutions and to evaluate its impact on business. It also highlights HCL’s capabilities in analyzing IT projects and implementing the CoQ approach across business domains, which bear ample testimony to the fact that CoQ is a business opportunity and not a cost centre. This paper will be of interest to IT professionals who are exploring ways to help their companies speed up time‐to‐market and cut operational cost.

Published in: Business, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

HCLT Research Paper: Cost of Quality

  1. 1. Cost of QualityNeeds Your Attention Not Sometimes But Always Authors Manas Chakraborty, Symu Koul, & Suresh Sundararajan, ETS-IV&VS HCL Technologies, Chennai Dr Usha Thakur, ATS Technical Research HCL Technologies, Chennai
  2. 2. CoQ: Needs Your Attention Not Sometimes But Always© 2010, HCL Technologies Ltd. June 2010Cost of Quality: Needs Your Attention Not Sometimes But Always Page 2
  3. 3. ContentsIntroduction ............................................................................................................................................................ 4Purpose ................................................................................................................................................................... 4CoQ and Its Relevance ............................................................................................................................................ 5CoQ Service from HCL ............................................................................................................................................. 7 CoQ as an Integral Part of SDLC ........................................................................................................................ 12Benefits ................................................................................................................................................................. 14Cost of Quality: Needs Your Attention Not Sometimes But Always Page 3
  4. 4. IntroductionIn and of themselves tools and technologies are not very useful and when they end up in the hands of peoplewho do not understand their significance, they are as good as not having any. By the same token a softwaresolution is not very valuable in itself, but it can serve as a powerful tool for enabling and transforming businessoperations; in fact, it has the potential of giving impressive business results.Of late, IT roles, activities, solutions, and services have come under the scanner of business executives and arebeing assessed for the extent to which they create (business) value i.e., help company executives in achievingstrategic objectives such as giving their company a competitive edge, increasing profit margins, cutting overalloperational costs, retaining existing customers and obtaining new ones, increasing customer satisfaction,improving internal communication, and speeding up time-to-market for software products and services. It is,therefore, not surprising that the impact of software solutions and services is increasingly being measured interms of business criteria such as Return on Investment (ROI) and Return on Quality (ROQ). Moreover,businesses are no longer willing to work within the confines of IT silos; they want IT to adapt easily to thedynamic needs of business without compromising on quality.If the goal of software solutions is to improve the way a business is run so that it becomes more profitable andhealthy, then Cost of Quality (CoQ) is a key approach for achieving that kind of improvement. The principles ofCoQ are well established in the manufacturing industry, given its long history.1 Their relevance in determiningthe quality of software solutions has been understood only in the last couple of decades, mostly in the contextof the costs incurred as a result of delivering poor quality software. The main idea behind a CoQ approach is toincrease revenue and decrease operational cost by reducing rework and maximizing opportunities.PurposeThe purpose of this paper is to illustrate the significance of incorporating CoQ principles in software solutionsand to evaluate its impact on business. It also highlights HCL’s capabilities in analyzing IT projects andimplementing the CoQ approach across business domains, which bear ample testimony to the fact that CoQ isa business opportunity and not a cost centre. This paper will be of interest to IT professionals who areexploring ways to help their companies speed up time-to-market and cut operational cost.1 For an interesting read on this see, Matthew Littlefield, "The Cost of Quality: Benchmarking Enterprise QualityManagement," (2007) http://www.hertzler.com/php/portfolio/white.papers.php [April 2010]->represents when the sitewas accessed.Cost of Quality: Needs Your Attention Not Sometimes But Always Page 4
  5. 5. CoQ and Its RelevanceAny discussion on the subject of CoQ leads one to acknowledge the contributions of Juran, Gryna and Crosby2who emphasized that it is more economical to pay attention to quality early in the lifecycle of a product (byincurring prevention and appraisal costs) than to wait for products to fail (for non conformance) and then fixthem (thereby incurring failure costs). Realizing its significance, the Software Engineering Institute (SEI) hasmade quality an integral part of its Capability Maturity Model Integration (CMMI) best practices.3 Furthermore,many organizations have made an effort to improve the quality of their software by integrating the principlesof Six Sigma.4According to Crosby,  Quality means conformance to requirements, not goodness  Quality is achieved by prevention, not appraisal  Quality has a performance standard of Zero Defects, not acceptable quality levels  Quality is measured by the Price of Non-conformance™, not indexes5In other words, CoQ refers to the cost that companies undertake to rectify the mistake they made and have tocorrect it. By the same token CoQ refers to the cost companies incur to ensure that the software solutions theydeliver to their clients are free of defects. To achieve that goal, companies need to incur costs (read ‘invest’) inthe form of resources, tools and activities throughout the lifecycle of software to identify and prevent defectsas well as potential failures. The seriousness of problems arising from software errors was very well highlighted2 For their earliest contributions the CoQ debate see J. Juran and F. Gryna, Quality Control Handbook 4th ed., McGraw-Hill, New York, 1988. See also P. Crosby, Quality Is Free: The Art of Making Quality Certain McGraw-Hill, New York, 1979.3 For an implementation of quality principles, see Dennis R. Goldenson, et al., "Why Make the Switch: Evidence about theBenefits of CMMI," (2004) http://www.sei.cmu.edu/library/assets/evidence.pdf [April 2010].4 While adherence to Six Sigma principles in software has its share of critics, they have been used for improving thequality of software and reducing the overall CoQ. See, for instance, Dr. Bill Eventoff, "Applying Six Sigma to SoftwareDevelopment: A Practical Guide," (2002) http://www.swqual.com/SQGNE/presentations/2004-05/Eventoff%20May%202005.ppt [April 2010]. See also Maneesh Aggarwal, "Six Sigma Meets Software Development,"http://www.isixsigma.com/index.php?option=com_k2&view=item&id=1307:&Itemid=49 [April 2010].5 Cited in Kevin Weiss, "Biography - Philip B. Crosby: June 18, 1926 – August 18, 2001,"http://www.philipcrosby.com/25years/crosby.html [April 2010]. For an interesting article on calculating the ROI of CoQ,see Sandra A Slaughter, et al., "Evaluating the Cost of Software Quality," COMMUNICATIONS OF THE ACM August1998/Vol. 41, No. 8. This paper is also available at http://www.cse.buffalo.edu/~hungngo/SCE-Papers/p67-slaughter.pdf[April 2010].Cost of Quality: Needs Your Attention Not Sometimes But Always Page 5
  6. 6. by a study commissioned by the Department of Commerces National Institute of Standards and Technology(NIST) in the US in 2002.6 The study acknowledged that [s]oftware is error-ridden in part because of its growing complexity. The size of software products is no longer measured in thousands of lines of code, but in millions. Software developers already spend approximately 80 percent of development costs on identifying and correcting defects, and yet few products of any type other than software are shipped with such high levels of errors. Other factors contributing to quality problems include marketing strategies, limited liability by software vendors, and decreasing returns on testing and debugging, according to the study. At the core of these issues is difficulty in defining and measuring software quality. The increasing complexity of software, along with a decreasing average product life expectancy, has increased the economic costs of errors. The catastrophic impacts of some failures are well-known. For example, a software failure interrupted the New York Mercantile Exchange and telephone service to several East Coast cities in February 1998. But high-profile incidents are only the tip of a pervasive pattern that software developers and users agree is causing substantial economic losses.7In 2008 IDC conducted a survey among quality managers of 139 North American companies. The surveyrespondents estimated that if 100% of software defects were addressed and remediated prior to production,they would experience a 32% cost savings.8 In 2008 Original Software conducted a qualitative opinion survey ofsenior IT executives and CIOs from organizations with average annual revenue of $5.8billion and an average ITbudget of $764 million. The survey revealed that over 60% of them felt a strong need to improve the quality ofsoftware applications.96 See Michael Newman, "Software Errors Cost U.S. Economy $59.5 Billion Annually: NIST Assesses Technical Needs ofIndustry to Improve Software-Testing," (2002) http://www.nist.gov/public_affairs/releases/n02-10.htm [April 2010]. Seealso Bill Hoffman, "Battling Software Defects One Developer at a Time," (February 23, 2010)http://www.itbusinessedge.com/cm/community/features/guestopinions/blog/battling-software-defects-one-developer-at-a-time/?cs=39611 [April 2010].7 Ibid. Similar concerns were highlighted by Herb Krasner. His research led him to conclude that "[t]ypical manufacturingCoQ, rang[es] from 5 to 25 percent of company sales, [which] contrasts significantly with CoSQ. With the present state ofsoftware engineering practice we can expect CoSQ to range from 10 to 70 percent of development costs. Even accountingfor the margin between production costs and sales, CoSQ appears to be roughly twice manufacturing CoQ." See HerbKrasner, "Using the Cost of Quality Approach for Software," (February 2006), p. 7,http://lifelong.engr.utexas.edu/pdf/sqi/cosq-article-new06.pdf [April 2010].8 See Melinda-Carol Ballou, "Improving Software Quality to Drive Business Agility," (June 2008)http://www.coverity.com/library/pdf/IDC_Improving_Software_Quality_June_2008.pdf [April 2010]. This survey wassponsored by Coverity Inc.9 See "Software Quality and Testing: A CIO Perspective," http://www.origsoft.com/whitepapers/ [April 2010].Cost of Quality: Needs Your Attention Not Sometimes But Always Page 6
  7. 7. CoQ Service from HCLHCL’s CoQ service is about total quality management of software in the pre or post development phase. It ismost helpful in determining the structure of cost across the lifecycle of software and in developing andimplementing best practices strategies. This service includes:  Thorough assessment of Quality Assurance (QA) activities and cost structure in four areas: planning, engineering, review and testing, and rework  Detailed gap analysis between corporate vision and existing QA practices in relation to where the organization concerned wants to be  Root-cause analysis of both costs: good and poor quality  Recommendations for improving CoQ  Identification of CoQ business cases  Implementation of high quality standardsAs illustrated in Figure 1, a good part of the CoQ exercise is contingent upon parameters used for determiningthe quality of ‘good’ and ‘poor’ quality. Figure 1: Cost of Quality ParametersCost of Quality: Needs Your Attention Not Sometimes But Always Page 7
  8. 8. Cost of Good Quality: Appraisal activities, among others, measure, evaluate and audit products or services toassure conformance to quality standards and performance requirements; their aim is to ensure that softwareis free from defects and conforms to customer requirements. Prevention activities ensure that necessarytraining, processes, tools, and hardware as well as resources are available for performing appraisal activities.Since both the activities are pre-emptive measures, they are regarded as the cost of good quality.Cost of Poor Quality: When detects (arising, for instance, from non-conformance to customer requirements)are not caught during the development phase of software and are noticed just prior to delivery, they require agreat deal of re-work and re-testing effort. If software defects come to the surface after delivery, the cost ofreview and root-cause analysis effort are significantly higher. Irrespective of when the failures come to light(before or after delivery), they both reflect non-conformance to customer requirements, and are hencereferred to as cost of delivering poor quality software.The value of being conscious about the cost of good and poor quality is fully realised only when, as shown inFigure 2, the assessment data obtained from various stakeholders (at a client organization) is entered in HCL’sCoQ analysis tool. Figure 2: CoQ Modeling ResultsFurthermore, as illustrated in Figure 3 and Figure 4, the HCL analysis tool allows for the quantification andconsolidation of quality parameters. Consequently, senior management at client organizations can then focuson areas that require further improvement and develop a strategy for keeping the CoQ at acceptable levels.Cost of Quality: Needs Your Attention Not Sometimes But Always Page 8
  9. 9. Figure 3: Quality Parameters Reflecting As-Is Situation Figure 4: Quality Parameters Reflecting a Reduction in CoQCost of Quality: Needs Your Attention Not Sometimes But Always Page 9
  10. 10. The HCL team uses an indexing system to make the results of the CoQ exercise meaningful to clientorganizations. For instance, in the sample shown in Figure 5, the index number of 4, (which represents lowquality and high cost) was arrived at following a cost and quality assessment of testing practices at a clientlocation. The results allowed senior management to identify areas that required attention in order to reducecost and improve software quality. Figure 5: CoQ IndexingBased on its root cause findings, the HCL CoQ team identifies initiatives for closing the gaps between As-Is andthe desired objectives. As shown in the sample in Figure 6, the proposed initiatives are clubbed into solutionthemes (group of initiatives), prioritized for ease of implementation, and then mapped to an IT strategy. TheHCL team also calculates the business benefits of implementing its proposed solution and suggests numerousoptions for its implementation such as building the proposed solution in-house organically, partnering with athird party to obtain the required skills, developing skills with third party via a co-sourcing/out-sourcingstrategy, or out-sourcing the entire solution to another company.Cost of Quality: Needs Your Attention Not Sometimes But Always Page 10
  11. 11. Figure 6: Theme Based Solution PropositionAs illustrated in the sample in Figure 7, the HCL team prioritizes the themes in its proposed solution and mapsthem in a 2/2 matrix to show specific business benefits and ease of implementation. This exercise helps clientorganizations to focus on recommendations that are not only easy to implement right away but also offerbusiness benefits (in terms of quality, cost, and productivity). Moreover, as and when required, clientorganizations can extend the same recommendations to other projects in phases.Cost of Quality: Needs Your Attention Not Sometimes But Always Page 11
  12. 12. Figure 7: Business Benefits of Proposed SolutionCoQ as an Integral Part of SDLCAlthough it is widely acknowledged that CoQ principles and practices should be used throughout the lifecycleof software, most software teams find themselves walking a tightrope between the needs of business teams(for faster time-to-market) and development teams (who require more time for comprehensive testing).As shown in Table 1, in cases where development teams have the mandate to integrate CoQ principles andpractices throughout the lifecycle of software, HCL’s CoQ team can assist client organization in severalconcrete ways to overcome challenges that are faced by software, testing teams, and allied service teams.Cost of Quality: Needs Your Attention Not Sometimes But Always Page 12
  13. 13. Testing Phase-Wise Activities Business Benefits HCL Deliverables Challenges Phases to ClientPlanning Requirements analysis and Reduction in effort Reports Ensuring the implementation feasibility and cost to fix defects  Requirements analysis involvement of the (discovered later)  Tool evaluation CoQ team right from Identification of tools for test Documents requirements phase management, defect management, Savings realized on  Testing estimation test automation and performance tool cost and  Test strategy testing rationalization of  Microsoft project plan tools usage across  Non functional multiple projects requirements  Test scenario identification  Technologies and skills set identificationHigh Level Design analysis and feasibility study Savings in effort and Reports UnderstandingTest Design cost related to defect  Detailed design software functionality Identification of: fix at early stage feasibility analysis  impact to upstream and  High level test Ensuring the downstream application and scenario and impact involvement: aligning testing activities to analysis  Domain experts cover all areas Documents  Performance  test environment and test  Requirements engineering data set-up Traceability Matrix experts  Hardware and System capacity planning and load software modeling requirements and Test Data control sheet for gathering test dataDetailed Detailed test case preparation and Avoid spend on Upload test cases in test Having resources withTest Design review of test cases (by business rework management tools skills set in testing users/SMEs) to ensure complete techniques coverage of test cases against functionality Performance scenario identificationTest Smoke testing to ensure basic Launch of new Reports Having resources whoExecution functionalities are working such features /application  Test summary are adept at that the entire QA team can be within budget and on  Daily status addressing challenges involved for testing all the features time  Defect status related to test environment and test System integration testing, data set-up regression testing, user acceptance testing, performance testing, Identification of performance bottlenecks and recommendations fixes Script execution in coordination with other resources involved for monitoringCost of Quality: Needs Your Attention Not Sometimes But Always Page 13
  14. 14. Testing Phase-Wise Activities Business Benefits HCL Deliverables Challenges Phases to ClientSupport Defect verification and impact Increase in customer Documents Collecting data on analysis satisfaction resulting  ROI based automation  customer from better  Production defect satisfaction data Update test cases related to defect performance of trend analysis  revenue fix software generation  penalties paid Support for enhancement and Savings in effort and and support impact analysis cost due and increase charges in revenue due to Calculation of Return on readily available test Having resources with Investment ( ROI) based on automation advanced knowledge application readiness, number of framework and ease of test automation release and nature of of maintenance enhancements Customization of existing HCL test automation framework requiring minimal maintenance Table 1: CoQ and SDLCBenefitsIf software products suffer from a high number of defects, low quality, and require a great deal of manualeffort and rework, then it is next to impossible to achieve business objectives such as speed up time-to-marketand lower operational cost. What the CoQ approach from HCL allows client organizations to do is not onlyidentify problem areas (which result in poor quality of software) but also implement a holistic plan of actionfor achieving business objectives and monitoring its progress in a measurable manner on a continuous basis.Cost of Quality: Needs Your Attention Not Sometimes But Always Page 14