Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to take organizations to higher testing maturity suresh bose anagha mahajan_upload


Published on

For the past many years, the focus of the industry was on improving quality by writing better code and using automation and engineering. These efforts did show results but we are still far from an illusive zero-defect state while fulfilling expected time to market.

In order to further boost software quality improvement efforts, the focus has now shifted to improving testing processes. Test Maturity Model integration (TMMi) is a detailed model for test process assessment and improvement. TMMi staged model contains stages or levels that determine the maturity of an organization and lays down roadmap for moving from one maturity level to next. As the processes evolve, an organization passes from adhoc (L1) to Optimized levels (L5). Application of TMMi will lead organizations to highly mature test process, reduced rework (about 10%), increase in defect removal efficiency (about 85%), higher productivity (about 10 test cases per day or 15% higher) and faster time to market.

Learning Objectives:
1. How can your organization improve the effectiveness of testing
2. How can you sustain testing maturity
3. What is the improvement roadmap for the implementation journey to improve testing processes
4. Why TMMI is preferred to CMMI for any testing organization

Published in: Technology
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ ◀ ◀ ◀ ◀
    Are you sure you want to  Yes  No
    Your message goes here

How to take organizations to higher testing maturity suresh bose anagha mahajan_upload

  1. 1. Suresh Chandra Ganesh Bose Anagha Mahajan HOW TO TAKE ORGANIZATIONS TO HIGHER TESTING MATURITY
  2. 2. Biography 2 Biography Suresh Chandra Bose, Ganesh Bose is a Manager Consulting at Cognizant Business Consulting practice. Suresh is based out of Austin, Texas and has been in the IT Industry for more than 16 years with vast Consulting experience in various industries and executed Strategic initiatives for various Fortune 100 companies in the areas of PMO, PPM, Process Consulting, Program Management, TMMI assessment/implementation, Organization Strategy, Test Consulting and CIO/Governance Dashboard/Metrics for various clients across the globe. Anagha Mahajan is a Senior Consultant with Cognizant’s Business Consulting practice. Anagha has diverse global experience with multiple Fortune 500 companies in Technology, Telecommunications and Healthcare. Her specialties include Due Diligence, Process Improvement, IT Transformation, IT Strategy, IT Governance and Organization Maturity Improvement. Anagha has an MBA from Babson college and is a Six Sigma Black Belt Certified professional.
  3. 3. Overview of TMMi Need for TMMi Assessment TMMi Assessment Maturity Rating Methodology Contents 3 Benefits of TMMi Process Transformation Case study Testing Maturity: Need of the hour Organizational Implementation Approach Critical Success Factors for Implementation
  4. 4. Everybody is not Michelangelo 4 Become Michelangelo first, for everyone else must follow process.
  5. 5. 5 Testing Maturity: Need of the hour Improves Time to Market Cost of Supplier Contracts Cycle Time Cost of Delivery Ability to plan Testing Standardizes Testing Method Implementation of Industry best practice Product Quality Defect tracking metrics Ease of integrating Test into the delivery model Process improvements Reduces Unpredictability Rework
  6. 6. 6  TMMi is developed by the TMMi Foundation  TMMi (Test Model Maturity Integration) is a detailed model for test process improvement and is positioned as being complementary to CMMi  Assessment is performed taking into consideration the level the Organization want to be appraised of  Contains guidelines and framework for test process improvement  Has five maturity levels for process evaluation in systems and software engineering  Addresses cornerstones of structured testing as lifecycles, techniques, infrastructure and organization  By far the most popular model in the Industry for benchmarking Testing practices TMMi Maturity Levels Overview of TMMi 1- Initial 2- Managed 3- Defined 4-Measured 5- Optimization
  7. 7. 7 Need for TMMi Assessment Standardized and Optimized testing process along with higher degree of process compliance Effective monitoring and control mechanism to mitigate schedule and cost overruns and quality issues Showing value to our customers as IT getting more complex Assess efficacy of support functions viz. Environment, Build Management etc. ,for testing activities Attesting the current testing process against Industry standards to enhance credibility in market place Test Performance goals with Metrics and Measures to predict performance and drive STLC improvements Integration of industry best practices with the testing process to achieve business objectives
  8. 8. 8 • Capability Maturity Model Integration • Developed by Carnegie Mellon University • Focused on end to end SDLC practices • Limited focus on Test Improvements • Has both staged and continuous representation • Has 3 others frameworks – CMMi for Development, CMMi for Acquisition and CMMi for Services • Test Maturity Model Integration • Developed by TMMi Foundation • Focused only on Testing processes/ practices • Limited focus on Non-Testing Improvements • Has only staged representation • No additional TMMi frameworks CMMI SVC CMMI DEV CMMI ACQ Comm on Core PAs Comparison - CMMi and TMMi
  9. 9. 9 5. Optimized •Test Process Optimization •Quality Control 4. Measured •Test Measurement •Product Quality Evaluation •Advanced Peer Reviews 3. Defined •Test Organization •Test Training Program •Test Life Cycle and Integration •Non-Functional Testing •Peer Reviews 2. Managed •Test Policy and Strategy •Test Planning •Test Monitoring and Control •Test Design and Execution •Test Environment 1. Initial •Testing is a chaotic, •Undefined process Continuous Process Improvement Process Standardization Stable Process Can Repeat Tasks Ad Hoc Informal Unpredictable Quantitatively Managed P r o c e s s M a t u r I t y Improvement Roadmap for Implementation
  10. 10. Assessment Plan Our Assessment Approach 10 Cognizant will employ its accredited method comprising of documentary evidence reviews and interviews to ensure adequate coverage of the generic and specific goals and practices across various process areas of TMMi, as illustrated below for Level 2 and 3 assessment. Timelines change based on the maturity assessment levels Engagement Planning RecommendationsTMMi Assessment Informal Assessment Survey (Optional) Documents Review Week A Week B Week C Identify Strengths & Improvement Opportunities Findings Report Recommendations & Implementation Plan - Deliverable Interviews Initial discussions to understand requirements Present assessment approach and understand objectives & constraints Agreement on Proposal and onboard resources Develop Proposal for Engagement Kick-off Determine Maturity levels Develop Recommendations & Implementation Plan Implementation Order & Priority Best practice sharing across projects/ applications Management Thoughts
  11. 11. TMMi Assessment Maturity Rating Methodology 11 Specific & Generic Goals RatingMechanism Test Policy & Strategy Test Planning Test Process Optimization Establish Test Policy Establish Test Strategy Institutionalize Defined Process…………. Process areas Specific&Generic Practices ProcessAreas Specific&Generic Practices Questionnaire /Client Survey Responses Interviews Documents Review Overall Process Area Maturity Level Enablers Inhibitors Instance Characterization Organizational Unit Characterization Goal Level Rating Maturity Levels Calibration …………. Specific&Generic Goals * Ratings represented above are Illustrative Only
  12. 12. TMMi Maturity Level Calibration Guidelines Process Areas are designated the appropriate maturity levels based on the following guidelines  Fully Achieved -  Convincing evidence of process compliance  Systematic and widespread implementation of process  No obvious weakness in in distribution, application and results of this process exists  process achievement is between 85 and up to 100%  Largely Achieved –  Significant evidence of process compliance  systematic and widespread implementation of process  Minor weakness in distribution, application and results of this process exists  process achievement is between 50 and up to 85%  Partially Achieved –  Some evidence of process found  Process exhibit significant weaknesses, be incomplete, not widespread, or inconsistent in application or results.  process achievement is between 15 and up to 50%  Not Achieved -  Little or no evidence of process  Process achievement is between 0 and up to 15%  Not Rated - Any supporting goal is Not Rated  Not Applicable - The process area is considered not to be in the scope of the assessment or applicable to the organizational unit by the Lead Assessor 12
  13. 13. Improving and Sustaining Testing Maturity 13 Laying the ‘Foundation’ Initiate ‘Transformation’ Drive the recommendations aligned with desired QA outcomes, measure effectiveness and stabilize Prioritized ‘Optimization’* Quality Policy Test Policy Test Approach Governance Communication Responsibilities Implementation of testing capability recommendation Implementation of tailored processes in projects Streamline key Functions Pilot key testing recommendations in identified projects Refine test process improvements (if required) Standardize testing at organization level *Review Checkpoint – evaluation of effectiveness of implemented recommendations to assess and take corrective actions/refine if required Focus on transforming the key focus areas and kick start rolling out processes and testing enablers Implement the foundation layer focussing on setting up test Function and Lifecycle Implementation Foundation TransformationOptimization
  14. 14. TMMi Process Transformation - Projected Savings 14 A high level snapshot of the improvements in specific process areas have been categorized into the following themes and levers, whose benefits and saving potential have been highlighted in the subsequent slides Level 1. Initial 2. Managed 3. Defined 4. Measured 5. Optimized Savings & ROI ROI over previous Maturity Level 0.0% 26.1% 40.1% 13.8% 8.2% Total Return on Investment % 0.0% 26.1% 55.7% 61.8% 64.9% End to End Process Framework Quality Assurance Strategy • Well defined quality gates • Quality assurance governance • End to End validation life cycle processes • Clear responsibility and accountability for test delivery Validation Risk Profiling Process Maturity • Implementation and institutionalization • Compliance evaluation • Risk and impact tracking • Preventive measures Continuous Improvement • Causal Analysis • Incremental improvements
  15. 15. Benefits of TMMi Process Transformation TMMi Levels Improvement Areas Cost Quality Speed to Market 2. Managed Test Policy and Strategy Test Planning Test Monitoring and Control Test Design and Execution Test Environment 3. Defined Test Organization Test Training Program Test Life Cycle and Integration Non-Functional Testing Peer Reviews 4. Measured Test Measurement Product Quality Evaluation Advanced Peer Reviews 5. Optimized Test Process Optimization Quality Control
  16. 16. 16 Critical Success Factors for Implementation Sponsor • Continued commitment and focus from Project sponsor and Executive management • Attention and support of all Stakeholders (including Business, Development, Testing) Participation • Active participation of practitioners throughout the engagement Feedback • On-going support and feedback from all members of the organization Communication • Appropriate planning and communication
  17. 17. 17 Case study: TMMi assessment and implementation for a major technology company “Conduct assessment of current testing processes against TMMi level 3 maturity, build and implement transformation roadmap that addresses the gaps between current and target end state - taking into consideration tactical and strategic opportunities to achieve level 3 maturity level”  Cognizant was invited to carry out a consulting engagement to: o Benchmark current test processes against TMMi Level 3 maturity o Identify the target test process  Based on the findings, Cognizant defined a roadmap addressing the gaps and also recommended a phased approach to improve the testing process with an objective of transforming the client to highly mature testing org DELIVERABLES:  Current State Report – Detailed overview of the Current strengths and best practices & Maturity scoring by focus groups within each IT groups  Target State Report with Implementation Roadmap - Overall solution summary & High-level recommendations across focus areas & Implementation approach, roadmap along with timelines  Testing Process Artifacts – Test Policy and Strategy, TCP, Test Design Techniques, Traceability, Role Skill matrix, Test Environment, NFT, Risk Management and Defect Management.  Cognizant proposes a phased approach to transform client’s validation group to TMMi Level 3 Maturity. Wherein first key process improvement capabilities first defined and then implemented in an Agile/continuous fashion  Key Solution Themes identified for achieving improvements are: o Organization – Setup audit function, NFT team and configuration team. o Lifecycle and Processes – Refine test planning, employ test design techniques, maintain bi-directional traceability and utilize TCP for estimation o Tools, Infrastructure and Support Functions – Streamline risk management, formalize review process, facilitate standardized training and test environment management. BENEFITS SNAPSHOT  Team Size: 5 Consultants (2 onsite, 3 offshore)  Duration: 4 weeks (assessment) + 12 weeks  Weekly status report shared with key stakeholders that details activities completed in current week, plan for next week, overall status and risks  Technical Working Group (TWG): A group of cross- functional individuals from client organization was formed to act as change agents, provide feedback and review and approve Cognizant artifacts  Train the trainers model: Training conducted for identified key resources on processes/capabilities and they will in turn train their respective team members ASSESSMENT HIGHLIGHTS ASSESSMENT APPROACH KEY RECOMMENDATIONS  Scalable organization with improved ability to roll out predictable quality outcomes  Standardized processes within the organization  Reduced defect leakage and quicker time to market  End to end metrics platform for unified reporting to stakeholders  Optimized automation / regression approach to increase overall application quality  Reduced cost of testing ownership  Improved levels of productivity
  18. 18. References 18 • TMMi Foundation Reference Model
  19. 19. Contact Us @ 19 Suresh Chandra Ganesh Bose Anagha Mahajan
  20. 20. 20 Questions?
  21. 21. 21