Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. The Future of Testing / QA Richard Bornet [email_address] 416-895-7176 www.scenariotester.com
  2. 2. Can we predict the future? <ul><li>Most people say no. </li></ul><ul><li>We can predict patterns? </li></ul><ul><li>Future is not just passive, we can affect the future! </li></ul>
  3. 3. History of World in One Slide <ul><li>You can predict the future. But you have to look at the long term big picture. </li></ul><ul><li>Stone Age -> Agriculture Age -> Industrial Age -> Information Age </li></ul><ul><li>The one common element across the ages is CHANGE and change is speeding-up. </li></ul><ul><li>The reason for change is innovation. </li></ul>
  4. 4. How Innovation Changes Work <ul><li>Home workers </li></ul><ul><ul><li>Manual </li></ul></ul><ul><ul><li>Cottage Industry </li></ul></ul><ul><li>Central Workplace </li></ul><ul><ul><li>Manual Work </li></ul></ul><ul><ul><li>Tools </li></ul></ul><ul><ul><li>Assembly line for component assembly </li></ul></ul><ul><ul><li>Robots / Automation </li></ul></ul><ul><li>Unit Cost Decreased. </li></ul><ul><li>Fewer individuals involved in creating the product. </li></ul>
  5. 5. So Where Are We in QA <ul><li>Manual with tools </li></ul><ul><ul><li>Word </li></ul></ul><ul><ul><li>Excel </li></ul></ul><ul><ul><li>Defect-tracking systems </li></ul></ul><ul><ul><li>Requirements managers </li></ul></ul><ul><ul><li>Test automation </li></ul></ul><ul><li>Change in QA will follow traditional growth patterns. </li></ul><ul><li>Future Goal </li></ul><ul><ul><li>Much higher productivity and efficiency. </li></ul></ul><ul><ul><li>Significantly improved ROI </li></ul></ul>
  6. 6. Return on Investment <ul><li>ROI will be the central driving force behind IT budgets in next decade. </li></ul><ul><li>QA will have to show a +ve ROI. </li></ul><ul><li>“ Business units will control virtually all discretionary IT spending. ” </li></ul><ul><li>Gartner </li></ul>
  7. 7. Return on Investment <ul><li>“ Software defects are costing the U.S. economy 60 billion dollars.” National Institute of Standards and Technology </li></ul><ul><li>“ It is much more cost effective to catch a bug in test than it is in production. About half the cost.” </li></ul><ul><li>“ Quality must be raised without incurring additional costs.” </li></ul>
  8. 8. Moving to an improved ROI <ul><li>Organizations are employing strategies to increase ROI. </li></ul><ul><li>If catching the defect in QA is more cost-effective than finding it in production then: </li></ul><ul><ul><li>Finding it before QA will be even more cost effective </li></ul></ul><ul><ul><li>Not creating it in the 1 st place will be preferred. </li></ul></ul>
  9. 9. So how is QA going to change? <ul><li>We Predict </li></ul><ul><ul><li>More testing by End user 5% </li></ul></ul><ul><ul><li>More testing by Developers 5% </li></ul></ul><ul><ul><li>Significantly shorter development cycles 7% </li></ul></ul><ul><ul><li>More knowledgeable QA Resources 7% </li></ul></ul><ul><ul><li>Increased appreciation for QA 8% </li></ul></ul><ul><ul><li>Improved Processes 14% </li></ul></ul><ul><ul><li>More Test Automation 31% </li></ul></ul>
  10. 10. Move to Packaged Software <ul><li>Less in-house development. </li></ul><ul><li>Buy packages which will execute the functions </li></ul><ul><ul><li>SAP, BPCS , PeopleSoft. </li></ul></ul><ul><li>Fewer and larger vendors. </li></ul><ul><li>Customize for specific business. </li></ul>
  11. 11. What this means for QA <ul><li>Vendors provides a stable application. </li></ul><ul><li>Vendors thoroughly test application. </li></ul><ul><li>QA will need to do less testing. </li></ul><ul><li>Test </li></ul><ul><ul><li>A good regression </li></ul></ul><ul><ul><li>Custom adaptations </li></ul></ul><ul><ul><li>Concentrate on whether business needs are being met. </li></ul></ul>
  12. 12. Pre-built components <ul><li>Common model in hardware. </li></ul><ul><li>Not just objects. </li></ul><ul><li>Whole functional modules. </li></ul><ul><li>Storage facility for On Demand videos </li></ul><ul><ul><li>Database may come for IBM </li></ul></ul><ul><ul><li>Web Services through .Net </li></ul></ul><ul><ul><li>Logging and monitoring system through another company. </li></ul></ul>
  13. 13. What this means for QA <ul><li>Vendors provides stable modules. </li></ul><ul><li>Less testing. </li></ul><ul><li>Test </li></ul><ul><ul><li>A good regression </li></ul></ul><ul><ul><li>Code that is built in-house </li></ul></ul><ul><ul><li>Concentrate on whether business needs are being met. </li></ul></ul><ul><li>Don’t test the video management system, it should work. </li></ul>
  14. 14. Outsourcing <ul><li>Bring in a third party to develop and maintain your systems. </li></ul><ul><li>Outsourcers bring: </li></ul><ul><ul><li>Stable code </li></ul></ul><ul><ul><li>Libraries of re-usable code </li></ul></ul><ul><ul><li>Applications tested in various companies </li></ul></ul><ul><ul><li>Costs shared between clients </li></ul></ul><ul><ul><li>Faster turnaround. </li></ul></ul>
  15. 15. What this means for QA <ul><li>Outsourcer provides stable code. </li></ul><ul><li>Outsourcers can perform the QA functions. </li></ul><ul><li>If testing in house test less testing. </li></ul><ul><ul><li>Outsourcer responsible for providing solid code </li></ul></ul><ul><ul><li>Concentrate on whether business needs are being met. </li></ul></ul>
  16. 16. Offshore development <ul><li>Same as outsourcing except theoretically cheaper. </li></ul><ul><li>Again, you can demand stable and defect free code. </li></ul><ul><li>What this means for QA </li></ul><ul><ul><li>Less testing </li></ul></ul>
  17. 17. Move testing to the developers <ul><li>Some companies have moved testing responsibilities onto their developers </li></ul><ul><li>QA departments shut down or downsized </li></ul>
  18. 18. Move testing to the developers <ul><li>Problems </li></ul><ul><ul><li>Programmers over worked. </li></ul></ul><ul><ul><li>Programmers may not know the business. </li></ul></ul><ul><ul><li>Programmers hate to test. </li></ul></ul><ul><ul><li>Programmers have time pressures. </li></ul></ul><ul><ul><li>Person who build should not have sole responsibility for testing. </li></ul></ul><ul><ul><li>Programmers may not know how to create and execute thorough testing. </li></ul></ul>
  19. 19. Move testing to the developers <ul><li>Advantages </li></ul><ul><ul><li>Moves testing earlier in development cycle. </li></ul></ul><ul><ul><li>Cost effective to catch bugs earlier. </li></ul></ul><ul><ul><li>Catch bugs at development stage even more cost effective. </li></ul></ul><ul><ul><li>Programmers like to use tools. </li></ul></ul>
  20. 20. Move testing to the developers <ul><li>Extreme programming </li></ul><ul><ul><li>Programmers design and build in tests </li></ul></ul><ul><li>Large use of Unit Testing tools </li></ul><ul><ul><li>Programmers build their own tools </li></ul></ul><ul><ul><li>Many tools are available often for free. </li></ul></ul>
  21. 21. Test Automation <ul><li>Many organizations try to automate. </li></ul><ul><li>It has huge potential. </li></ul><ul><li>Many fail (78%). </li></ul><ul><li>Almost never is it complete testing. </li></ul><ul><li>To often it is used for Smoke Tests. </li></ul>
  22. 22. Traditional Development Cycle Courtesy of Borland
  23. 23. New Model
  24. 24. Efficiency Improvements - Define / Requirements Stage <ul><li>Huge cost savings by catching problems early </li></ul><ul><ul><li>$65 per defect at Requirements stage </li></ul></ul><ul><ul><li>$1000 to $3000 at System test </li></ul></ul><ul><ul><li>$10,000 or more in Production </li></ul></ul><ul><li>Test the requirements </li></ul><ul><ul><li>Proof read the requirements </li></ul></ul><ul><li>There are technologies available now that will: </li></ul><ul><ul><li>Give you 100 % test coverage </li></ul></ul><ul><ul><li>Automatically generate test cases </li></ul></ul><ul><ul><li>Put testers and developers on same page </li></ul></ul><ul><li>Better tools for management of requirements. </li></ul><ul><li>In the future </li></ul><ul><ul><li>Process will be enhanced, simplified and automated more. </li></ul></ul>
  25. 25. Efficiency Improvements - Design Stage <ul><li>Designers and Developers will build software mindful of how we are going to test it. </li></ul><ul><ul><li>Build tests into the application. </li></ul></ul><ul><ul><li>Build hooks for testing into application. </li></ul></ul>
  26. 26. Efficiency Improvements <ul><li>Development Stage </li></ul><ul><ul><li>Unit test automation </li></ul></ul><ul><ul><li>Automation </li></ul></ul><ul><li>Deployment Stage </li></ul><ul><ul><li>Automation </li></ul></ul><ul><li>Production </li></ul><ul><ul><li>Automation </li></ul></ul>
  27. 27. TEST AUTOMATION <ul><li>Single biggest change in QA, will be the amount and type of test automation </li></ul><ul><li>This will include: </li></ul><ul><ul><li>New types of testing </li></ul></ul><ul><ul><li>New tools </li></ul></ul><ul><ul><li>New strategies </li></ul></ul>
  28. 28. Test Automation – Current State <ul><li>Stage 1: Record and Playback </li></ul><ul><li>Stage 2: Write Code </li></ul><ul><li>Stage 3: Modularity </li></ul><ul><li>Stage 4: Data Driven Testing </li></ul>
  29. 29. Test Automation – Who Does What <ul><li>Stage 5: Teach the testers how to program. </li></ul><ul><li>Stage 6: Make the automation tools simpler </li></ul><ul><ul><ul><li>Tree view diagrams. </li></ul></ul></ul><ul><ul><ul><li>“ WinRunner Lite”. </li></ul></ul></ul><ul><li>Stage 7: Teach programmers the business. </li></ul><ul><li>Stage 8 : Bring in Automation experts. </li></ul><ul><ul><li>Hire a programmer. </li></ul></ul><ul><ul><li>Bring in an external consultants. </li></ul></ul><ul><ul><li>Create a test automation team. </li></ul></ul>
  30. 30. Test Automation – Current State <ul><li>None of these stages fit the way testers work. </li></ul><ul><ul><li>Having to code or handing off your work and responsibilities to a 3 rd party is not acceptable. </li></ul></ul><ul><li>Almost never is it complete testing. </li></ul><ul><li>No wonder many automation efforts fail. (78%). </li></ul>
  31. 31. Stage 9: Empower the TESTERS <ul><li>Give testers an interface to enter test cases / test scripts. </li></ul><ul><li>Press a button to execute them. </li></ul><ul><li>Needs to be simple and powerful. </li></ul><ul><li>Create test scripts before the application is even built. </li></ul>
  32. 32. Web Services <ul><li>Endorsed by Microsoft, Borland, Sun and IBM. </li></ul><ul><li>It is the ability to provide hooks to an application which can be utilized by external applications. </li></ul><ul><li>CIO like it because they can keep their legacy systems. </li></ul><ul><li>Users like it because they can get new and useful interfaces. </li></ul>
  33. 33. Web Services – Testing <ul><li>Multiple Inputs </li></ul><ul><ul><li>GUI application, Palm Pilot, your phone, </li></ul></ul><ul><ul><li>your TV, Voice, special devices </li></ul></ul><ul><li>Testing will be more interesting as new forms of accessibility are developed. </li></ul><ul><li>Component model of software. </li></ul>
  34. 34. Web Services – Testing <ul><li>Web Services creates a stable environment. </li></ul><ul><li>Testing components. </li></ul><ul><li>Testing is split between </li></ul><ul><ul><li>Testing the web service </li></ul></ul><ul><ul><li>Testing the GUI / application </li></ul></ul><ul><ul><li>Testing the business rules. </li></ul></ul><ul><li>New simple tools. </li></ul>
  35. 38. New Model
  36. 39. FUTURE: Building the interface between the tests and the app. <ul><li>Currently you Point and Click </li></ul><ul><li>Building the scripts automatically </li></ul><ul><ul><li>Screen scrapers </li></ul></ul><ul><ul><li>Web services test tools will pick up the WSDL file. </li></ul></ul>
  37. 40. FUTURE: Building the interface between the tests and the app. <ul><li>Design Stage </li></ul><ul><ul><li>Developers when they design application, design hooks into the application to allow for data to be passed to and from the app. </li></ul></ul><ul><ul><li>Design to expose the application to an outside test tool. </li></ul></ul><ul><ul><li>Generally straightforward. </li></ul></ul>
  38. 41. FUTURE: Building hooks to test automation into the application <ul><li>Application arrives automatically automated. </li></ul><ul><li>Test tool automatically connects test cases and app. </li></ul><ul><li>Build extra scripts in VB and Java and compile code. </li></ul><ul><li>Same cost no matter how many testers. </li></ul><ul><li>Cheaper /more control. </li></ul>
  39. 42. FUTURE: Building testing right into the application. <ul><li>Hardware manufacturers have been doing this for ages. </li></ul><ul><li>In Production we call it monitoring. </li></ul><ul><li>eBay. </li></ul>
  40. 43. Future of QA <ul><li>Test in Production </li></ul><ul><ul><li>Automatic monitors for functionality and performance. </li></ul></ul><ul><ul><li>Programs, which can execute tests both automatically and on request. </li></ul></ul><ul><li>Same set-up can be used at Deployment and Development stage </li></ul>
  41. 44. Future of QA <ul><li>Get requirements right by thoroughly testing them. </li></ul><ul><li>Automating the writing of the test cases and test scripts. </li></ul><ul><li>Automated environment for testing </li></ul><ul><ul><li>Automatic monitors for functionality and performance. </li></ul></ul><ul><ul><li>Programs, which can execute tests both automatically and on request. </li></ul></ul><ul><li>Deployment testing is automated. </li></ul><ul><li>Production is automatically monitored and tested </li></ul><ul><li>Automated system allows for Troubleshooting and Diagnostics. </li></ul>
  42. 45. Future QA – Solves problems <ul><li>Increased ROI. </li></ul><ul><ul><li>Decreased costs. </li></ul></ul><ul><ul><li>Complete test coverage. </li></ul></ul><ul><ul><li>Faster turnaround. </li></ul></ul><ul><li>We need to reinvent ourselves or we will become obsolete. </li></ul><ul><li>Huge opportunities for QA. </li></ul>
  43. 46. New Model
  44. 47. Contact Information <ul><li>Richard Bornet </li></ul><ul><li>416-895-7176 </li></ul><ul><li>[email_address] </li></ul><ul><li>Web site </li></ul><ul><li>www.scenariotester.com </li></ul><ul><li>Presentation </li></ul><ul><li>www.scenariotester.com and go to Articles </li></ul>