Test Engineering Maturity Model- Impetus White Paper


Published on

For Impetus’ White Papers archive, visit- http://www.impetus.com/whitepaper

The paper talks about how Test Engineering Maturity Model helps companies save on time and costs, while improving the overall quality of the product.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Test Engineering Maturity Model- Impetus White Paper

  1. 1. The Test Engineering Maturity ModelW H I T E P A P E RAbstractRecent software testing research indicates that under pressureto meet deadlines, the testing teams often lose focus on the‘business’ goals expected from the activity.This white paper talks about the Test Engineering MaturityModel (TEMM) that Impetus Technologies has developed. TEMMhelps clients meet software testing goals while keepingcontinuous focus on business needs. The paper talks about howthis framework helps companies save on time and costs, whileimproving the overall quality of the product.The five levels of the TEMM have been described and explained.The white paper also talks about the benefits of each level, andhow they impact the testing function.Impetus Technologies, Inc.www.impetus.com
  2. 2. The Test Engineering Maturity Model2IntroductionFor most companies, software testing is an arduous and expensive activity.Organizations are often faced with a lack of time and resources, which limitstheir ability to test each and every line of code during the test execution cycle. Ittherefore becomes imperative for them to optimally utilize the limitedresources available within time-to-market constraints, to deliver better qualityproducts.The size of regression test suites grows as software evolves. Their size grows tosuch an extent that regression testing alone can account for as much as 50percent of the cost of software maintenance. Due to various constraints, it isnot possible to test each and everything evolving in a software product.Typically, such testing increases the cost and time associated with measuringquality and stretches the release cycle without providing an adequate Return onInvestment (ROI). To meet business deadlines and measure product quality,stakeholders use cost and time as the main driving factors. It is often seen thatcompanies try to measure quality based on cost and time, which leads tounsuccessful testing endeavors. While a huge amount is spent on testing, theproduct outcomes are not up to the mark and make companies feel that testinghas lost track, somewhere.There is a clear relationship between the three elementary forces i.e. Time,Cost, and Quality in any project. Time denotes the available time to deliver theproject, cost represents the amount of money or resources available and qualityrepresents the fit-to-purpose that the project must achieve to be a success.A ‘product’ is always created to meet ‘business needs. Business is the drivingfactor that decides on parameters such as time, cost, and quality. However, asthe delivery pressures increase in a release cycle, and companies focus on costand time to achieve a predefined quality, the concept of business loses shine.It is therefore important to ensure that business is not ignored in a testingenvironment. Industry is demanding a comprehensive testing package, apackage that can help in balancing between business, cost, quality, and time.What is required then is a framework that takes into account the four pillarsassociated with a product, and how these work in relation to each other.Impetus Technologies’ testing practice offers comprehensive end-to-end testengineering solutions in order to reduce the product development costs,improve quality, and reduce time-to-market. It has developed the TestEngineering Maturity Model, which helps in showing a definite direction forcomplete product testing.
  3. 3. The Test Engineering Maturity Model3The Key Objectives of TEMMThe TEMM focuses on the following:• Resolving the ambiguities arising during software product testing• Providing a definite direction to attain higher ROI in product testing bykeeping business goals and risks within sight• Defining the right set of Key Testing Activities (KTAs)• Providing a matrix with business, cost, quality and time as the primarycomponents• Offering five levels, which serve as benchmarks for any time of testing• Enabling organizations to opt for a desired level, based on the theirproduct testing requirementsOverall, the Test Engineering Maturity Model, provides companies with a set ofguidelines for improving their software testing.TEMM defines Key TestingActivities, popularly known as KTAs that helps in providing a definite direction tocompanies for their testing endeavors and helps in attaining a higher Return onInvestment (ROI). The model removes the need for organizations to reinvent thewheel. TEMM provides incremental and non-linear benefits to the teams thatwant to mature their testing activities. All they need to do is select the levelthey want to achieve, based on their business requirements and follow theactivities that are defined.
  4. 4. The Test Engineering Maturity Model4TEMM StructureTEMM Reference TableTEMM reference table can be used to get quick information on levels scope,outcome and limitations. Depending on the requirement of the project and thematurity level that any team wants to achieve, an appropriate level of TEMMneeds to be selected. Each level defines the key testing activities that will helpthe team to achieve the desired level. Identifying the layer and implementingthe activities will provide incremental benefits to the product testing teams,level over level.
  5. 5. The Test Engineering Maturity Model5Level Type Scope Outcome LimitationsIllusive CrudeproducttestingAd-hoc Test coverage based ontime available beforereleaseProduct quality mainlydecided based on gut feelingof test teamDomain/Applicationknowledge resides inindividualsTesting activities are persondrivenHigh probability of criticalbugs getting leaked intoproductionPrimitive ElementaryproducttestingManual-UI, Functional,Integration, System,UATAd-hoc - Non-FunctionalBasic manual testing isplanned and performed onthe productWell defined test cases forthe covered areasTesting on differentenvironments is acumbersome taskLonger testing cyclesNon-Functional aspects ofthe application are notcompletely testedHigh probability of criticalbugs getting leaked intoproductionReflexive EssentialproducttestingManual- UAT and Non-FunctionalClassic Automation-Functional and WhiteboxComplete, planned manualtesting is performed on theproduct that includesfunctional and non-functional testingUAT is planned andexecuted beforehand,minimizing surprises atlater stagesPrioritization helps inminimizing defect Leakageinto productionAutomation remainssecondary so all the benefitsof automation are notreapedLack of automation strategyresults in high maintenanceof test scriptsRequires longerimplementation time
  6. 6. The Test Engineering Maturity Model6Systematic StreamlinedproducttestingManual- Functional andStrategic Non-FunctionalPlanned Automation-Functional and Non-FunctionalCompletely planned Testingactivities, both manual andautomationRepeatable tests ondifferent versions of anapplication, even if theapplication changes isenhancedReduced regression testcyclesIncreased test coverageUnattended execution oftest suitesOptimal utilization ofresourcesThe testing metrics do notgive complete view of theproduct statusComprehensive ProgressiveproducttestingEnd-to-End (E2E)AutomationEngineering to Testing(E2T) AutomationContinuousImprovementAutomation of majoractivities involved inproduct testingAutomation of End-to-endbusiness scenariosContinuous Integrationhelps in detecting defectsearly in the life cycleVisibility into completeproduct status from boththe engineering and testingperspectiveContinuously look for waysto improve the currentpractices
  7. 7. The Test Engineering Maturity Model7SummaryTo minimize testing nightmares and enable the software product testing processto achieve maturity, the thumb rule is to revisit business needs at every stage.The participation of business at every step in the testing cycle ensures that thecritical factors are balanced and producing positive results that meet businessobjectives. Cost, quality, time, and business are the inseparable axes that willcertainly lead to the path of productive testing.By implementing TEMM, companies can quickly meet their desired qualityobjectives, while focusing on critical business needs. Using the designed path,that touches each and every aspect of the testing cycle, TEMM enablesenterprises to mature their testing activities, keeping process implementationworries behind.As organizations move from Level 1, through Level 5, the testing activities willyield higher benefits in terms of quality and business improvement—whilekeeping costs and time at a minimum.Impetus Technologies’ TEMM assessment methodology helps analyze andreview the testing progress, and provides updates on the exact level achievedand the requirements for reaching the next stage.About ImpetusImpetus is a Software Solutions and Services Company with deeptechnical maturity that brings you thought leadership, proactiveinnovation, and a track record of success. Our Services and Solutionsportfolio includes Carrier grade large systems, Big Data, Cloud,Enterprise Mobility, and Test and Performance Engineering.Website: www.impetus.com | Email: inquiry@impetus.com© 2013 Impetus Technologies, Inc.All rights reserved. Product andcompany names mentioned hereinmay be trademarks of theirrespective companies. May 2013