Automate virtualize and smart test the new testing realities

2,656 views

Published on

Presentation on changes occurring in the testing space due. Presented in Star East 2011.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,656
On SlideShare
0
From Embeds
0
Number of Embeds
340
Actions
Shares
0
Downloads
140
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Automate virtualize and smart test the new testing realities

  1. 1. Automate, Virtualize & Smart Test Understanding the new realities of Testing by Manoj Narayanan Director –Testing Services, Cognizant @manoj7698 http:everydaytesting.blogspot.com©2011, Cognizant
  2. 2. What does traditional testing offer? Discrete, independent Waterfall Methodology Requirements Architecture Developm Testing Release Gathering & Design ent Focused on defect identification 1
  3. 3. What does business want today? 24% 22% 17% 14% 13% 7% Reducing risk Quickly Reducing costs Delivering Improving Delivering 5% Compliance by preventing responding to by improving applications on collaboration applications and poor quality customer our time and on between before regulatory from impacting requests and development budget development competition issues satisfaction requirements and testing and QA productivity • 32% of organizations say they release software with too many defects • 38% of organizations believe they lack an adequate quality assurance program • 27% of organizations do not conduct any formal quality reviews2 More than 60% ©2011, Cognizant Source- QAI –Edista Testing
  4. 4. What does business want today? Reducing Risk Responding Fast 24% 22% 17% 14% 13% 7% Reducing risk Quickly Reducing costs Delivering Improving Delivering 5% Compliance by preventing poor quality responding to customer by improving our applications on time and on Beat Competitors collaboration between applications before and regulatory from impacting requests and development budget development competition issues satisfaction requirements and testing and QA productivity • 32% of organizations say they release software with too many defects • 38% of organizations believe they lack an adequate quality assurance program • 27% of organizations do not conduct any formal quality reviews3 More than 60% ©2011, Cognizant Source- QAI –Edista Testing
  5. 5. Traditional testing approach Discrete, independent Waterfall Methodology Requirements Architecture & Developmen Testing Release Gathering Design t Focused on defect identification 4 ©2011, Cognizant
  6. 6. We need to move testing to the beginning ofSDLC – focus on prevention than identification From Discrete and Independent Testing to… Requirements Architecture Development Testing Release Gathering & Design Waterfall Methodology …Continuous and Integrated Testing Scrum5 ©2011, Cognizant
  7. 7. The “Shift-Left” Philosophy Focus on Test Driven Development or a Non-functional variant testing Performance & Security PoCs Identify QA skill Early execution of QA, Automated end to sets needed Non-GUI testing and initial effort end regression estimate6 ©2011, Cognizant testing
  8. 8. Shift-Left Philosophy is accompanied by theneed for a new testing approach Unit effort Testing Unit Testing spent Percentage Service Layer Testing Service Layer Testing GUI Testing GUI TestingNeed to move away from traditional …to an approach focusing on increasedapproach focusing on GUI based testing.. defect capture through Unit and Service Layer Testing7 Automation becomes critical ©2011, Cognizant
  9. 9. This results in the need for seeding QualityEngineering into testing• A QE approach focused on testing components• A QA approach focused on regression and UAT QE Approach QA Approach8 ©2011, Cognizant
  10. 10. And “Smart Testing” dissolving boundaries oftraditional system & integration testing QE Approach QA Approach Regression Fast merging into a Unit Testing Testing social media and (automation) “crowd” aided Community Testing Service Layer Beta Testing Testing Functional User Testing Acceptance (automation) Testing9 ©2011, Cognizant
  11. 11. And “Smart Testing” dissolving boundaries oftraditional system & integration testing QE Approach QA Approach Regression Unit Testing Testing (automation) Service Layer Testing Community Functional Testing Testing (automation)10 ©2011, Cognizant
  12. 12. Leverage optimal mix of automation acrossthe lifecycle • Re-usable, automated test cases used by developers Unit Testing • Leveraging xUnit Frameworks • Leverage re-usable frameworks like RIDE, Fitnesse, CucumberManual Testing Service Layer Testing • Web Services testing • Leverage Service Virutalization • >90% automated regression test cases Regression Testing • 100% automated smoke testing • Continuous validation of test flow relevancy 11 ©2011, Cognizant
  13. 13. Prioritizing Automation: ROI driven approach Verify application paths that have high degree of frequency and other high volume activities where software failures would occur Validate applications with a high degree of risk which define or control the core of a company’s business; Failure can cause extreme disruptions in critical operations Any flow that needs to be run repeatedly in a build Repetitive Testing Automating applications that are or will be in production for a long time Greater benefits can be reaped from Test Automation if the test execution frequency is high Test Automation will be an ideal testing solution if the testing has to be performed on12 ©2011, Cognizant multiple environments
  14. 14. Prioritizing Automation: ROI driven approach Verify application paths that have high degree of frequency and other high volume activities where software failures would occur Validate applications with a high degree of risk which define or control the core of a company’s business; Failure can cause extreme disruptions in critical operations Any flow that needs to be run repeatedly in a build Repetitive Testing Automating applications that are or will be in production for a long time Greater benefits can be reaped from Test Automation if the test execution frequency is high Test Automation will be an ideal testing solution if the testing has to be performed on13 ©2011, Cognizant multiple environments
  15. 15. The need for early testing and automation alsotriggers environment related challenges Environment availability challenges – productivity losses, inability to duplicate COM App Database defects, performance testing Mainframe Ordering Swing App Business Process App Need for testing individual App Server Additional Apps components as part of “shift-left” EJB, WS, Now & Future ESB, Processing Web philosophy JMS, MQ App Database14 ©2011, Cognizant
  16. 16. Work Around: Virtualization & ServiceVirtualization UI Testing no longer the Environment availability challenges dominant testing method mitigated – simulate performance COM App Database environment, component transactions 5 Mainframe Ordering Swing App Re-usability increases, limited dependency on Business Process App development teams Additional Apps App Server Now & Future EJB, WS, ESB, JMS, Processing Web MQ App Early testing is now feasible. Defects identified at each Enables organization to adopt component level “Shift Left” testing15 ©2011, Cognizant Database LISA Virtualize Picture: Courtesy ITKO
  17. 17. Explaining Service Virtualization• Mainframe applications are consumed through queues – in this case, MQ• The MQ sends messages to the application which gets transformed through the adapter “Listener” ©2011, Cognizant
  18. 18. This has to be complemented by a centralizedTest Environment Management approach Cloud too is an option – even after the Amazon EC2 fiasco17 ©2011, Cognizant
  19. 19. And an organizational transition plan based oncurrent maturity level Stakeholder Discussions Evaluate Industry best practices Inputs Define Current QA Inputs Future State Artifact Analysis State Methodology AdoptedFocus on • Process & Methodology Focus on • Tools • Risk appetite • Infrastructure • Cost constraints • Operating Model Identify • Available time • People: Skill set/Roles implementat • Priority areas ion priorities • Current & Future state gap Stakeholder Validation Draw Inputs Implementation18 External dependencies Roadmap
  20. 20. But do not let the total cost creep up… • The QA organization needs to keep a constant watch on the Total Cost of Quality as it makes the transition Cost of Production Cost of Compliance Cost of Testing Defects Audits Total Cost of Quality • During the transition phase, the Cost of Testing goes up due to the increased focus in areas like skill ramp-up, automation and process re-alignment • The resultant increase in Total Cost of Quality needs to be continuously adjusted to ensure that it falls within best practice levels of 25–30% within 3 to 6 months of transition19 ©2011, Cognizant
  21. 21. And most importantly, do not forget tocommunicate • Ensure Executive & Stakeholder buy-in on the change • Establish a communication channel for regular updates and feedback20
  22. 22. Summary• Traditional testing approaches are being replaced by a “shift- left” approach to accommodate current business realities• Shift away from GUI based testing to component based testing• Focus on both QE and QA based testing – prevention rather than identification: Need for “Smart Testing”• Automation becomes critical to maintain the ever reducing testing window• New facilitators have risen: Community Testing, Service Virtualization, Test Environment Management• However, it is very critical that the organization has a well thought out transition plan to accommodate these changes21 And, DO NOT forget to communicate!
  23. 23. @manoj7698http:everydaytesting.blogspot.comwww.cognizant.com ©2011, Cognizant

×