Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

#ATAGTR2018 Presentation "Quality Engineering in DevOps Enabling Business Growth with seamless Production" By Prashant Yadav

12 views

Published on

Prashant Yadav from Amdocs has given us a lighting talk on "Quality Engineering in DevOps Enabling Business Growth with seamless Production" only at #ATAGTR2018.

please refer our linkedin post for session details
https://www.linkedin.com/pulse/lightning-talk-quality-engineering-devops-enabling-growth-alliance/

Published in: Technology
  • Be the first to comment

#ATAGTR2018 Presentation "Quality Engineering in DevOps Enabling Business Growth with seamless Production" By Prashant Yadav

  1. 1. Quality Engineering in DevOps : Enabling Business Growth with seamless Production - Prashant Yadav
  2. 2.  Agenda Traditional Testing Model- A Mere Bottleneck Amdocs Quality Engineering- Shift Left in DevOps Shift Left Enablers MTV Business Case Study Q&A
  3. 3. Information Security Level 2 – Sensitive © 2018 – Proprietary & Confidential Information of Amdocs3 Traditional Testing Model- A Mere Bottleneck System Test Functional Test Regression Test Business UAT E2E Test Integration Test GoLive Compatibility Test Accessibility Test Security Test Test Automation Test Automation Development Responsibility Testing Responsibility Revenue Assurance Unit Test Testing Responsibility Performance Test
  4. 4. Information Security Level 2 – Sensitive © 2018 – Proprietary & Confidential Information of Amdocs4 Amdocs Quality Engineering- Shift Left in DevOps System Test Functional Test Regression Test Performance Test Business UAT Supt E2E Test Integration Test GoLive Compatibility Test Accessibility Test Security Test Test Automation Test Automation Automation Nuggets Scrum Performance Test E2E Performance Test Compatibility Test Accessibility Test Security Test Continuous Progression Automation Continuous Regression Test E2EPerformanceTest BusinessUAT E2ESanityTest Consolidated Functional Test (Scrum Test + Integration Test) Unit Test
  5. 5. Shift Left Enablers
  6. 6. Information Security Level 2 – Sensitive © 2018 – Proprietary & Confidential Information of Amdocs Consolidated Functional Test (Scrum Test + Integration Test) Shift Left Enablers GoLive Automation Nuggets Scrum Performance Test Compatibility Test Accessibility Test Security Test Continuous Progression Automation Continuous Regression Test E2EPerformanceTest BusinessUATSupt E2ETest Development Responsibility Testing Core Responsibility 1 2 3 TA Automation One Team Iterative Scoping & Backlog Grooming
  7. 7. Information Security Level 2 – Sensitive © 2018 – Proprietary & Confidential Information of Amdocs7 Test Architect Role • Acceptance by Account Stake-Holders • 100 % MTV Driven Scope Break-Down • Non Functional Considerations For Each MTVs • MTV driven IA Sign-Off • Proactive Participation in Iteration Planning • Early Insight to Testing • Scrum Testing Quality Control
  8. 8. Information Security Level 2 – Sensitive © 2018 – Proprietary & Confidential Information of Amdocs8 Automation – Progression Automation • Benefits • Ability to run multiple execution cycles in CJ helping us to reduce escaping defects to UAT • Supporting Dormant CRs Testing before Go-Live • On-going Regression Suite enhancement • Challenges • Multiple Environment Configuration Demands additional effort for re-use • Unable to Stabilized Suite before CJ start due to time/people limitations
  9. 9. Information Security Level 2 – Sensitive © 2018 – Proprietary & Confidential Information of Amdocs9 Automation – Robotic CI & Code Coverage Analysis • Current Status • Unattended successful execution of extended sanity with nightly build (Ginger Integration With Jenkins) • 1st Successful Run of Regression Suite for Code Coverage Analysis using Jacoco • Approach & Benefits • Continuous enhancing scope of CI build sanity to ensure wide test coverage on nightly build. • Availability of Working Build Every Morning ( Currently limited to ST Env)
  10. 10. Information Security Level 2 – Sensitive © 2018 – Proprietary & Confidential Information of Amdocs10 One Team • Continuous Rotation of People Across Teams- Enabling CFT Enhancement • ITV Team Support in Portfolio Testing – Helped to Accelerate Progression Automation • Re-use of Automation- Continuous Enhancement in Regression Suite
  11. 11. Information Security Level 2 – Sensitive © 2018 – Proprietary & Confidential Information of Amdocs11 MTV – Breaking the bottleneck! SIT waits till the first PI which provides Testable features on integrated environment with Business value to the customerDevelopment System Testing SIT / E2E / UAT SIT Scope Agile To UAT Agile To ProductionBDD + TDD = MTV PI 01 Dev User Story Feature Epic CJ PI 02 Dev User Story Feature Epic CJ PI 03 Dev User Story Feature Epic CJ PI 01 Dev User Story Feature Epic CJ Scope SIT/E2E PI 02 Dev User Story Feature Epic CJ Scope SIT/E2E PI 03 Dev User Story Feature Epic CJ Scope SIT/E2E
  12. 12. Information Security Level 2 – Sensitive © 2018 – Proprietary & Confidential Information of Amdocs12 MTV Concept Project PI 1 5-7 Weeks Project PI 2 5-7 Weeks Project PI 3 5-7 Weeks Every PI starts from Scoping in which MTVs are defined Development & Testing are done continuously per MTV, enabling production deployment of each MTV Separately MTV ( Minimal Testable Value) An entity defined by the Test Architect during scoping, to describe the integrated scope element which the testers must receive in order to certify scope which carries business value and supports E2E tests MTV Example Data only mobile postpaid activation & usage for Residential non VPN subscribers Continuous Testing Testing activities are done in parallel at all test levels, certifying in each stage the scope of MTVs separately and moving them up to the next stage of testing. The certification is done at MTV level, allowing the customer to take the final decision if to approve the certified MTVs to production or wait for additional scope in the next delivered MTVs.
  13. 13. Customer Case Study
  14. 14. Information Security Level 2 – Sensitive © 2018 – Proprietary & Confidential Information of Amdocs14 NAM Customer Business Challenges 1. Continuous drops for progression MTVs 2. Manual intensive test execution 3. Resource optimization AQE Solution 1. High levels of automation & reusability 2. More testing coverage during progression 3. Automated production sanity 4. Unattended non-production sanity and CI 5. Automated regression run for weekly SWP ~110 flows being executed unattended on daily/while CI ~60% Regression 5 days 1 Day Customer Journey regression duration Reduction in Escaping Defects to UAT 22% 8% ~1250 TCs executed weekly via automation in progression & regression 40% Of defects found via automation 5-6 times Regression execution frequency Once Monthly ~15000-16000 TCs are executed during production sanity integrated Progression Automation MTV: Minimum Testable Value CI: Continuous Integration ~70%
  15. 15. Information Security Level 2 – Sensitive © 2018 – Proprietary & Confidential Information of Amdocs15 DevOps: NAM Customer • Testing driven MTV • Performance Shift Left • Feature level delivery • EPIC Certification • Robotic CI • Automation • Successful Customer Demos • One Quality authority • Minimal ENV • Automation reuse CI CD • MTV/Feature delivered weekly (3 to 7) • Further reduction in escaping defects • Earliest production stabilization • MTV/Feature level deployment • Continuous improvement • Automation reuse • Delivery KPI Dashboard
  16. 16. Q & A

×