Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

[DevDay2019] How AI is changing the future of Software Testing? - By Vui Nguyen, Software Test Engineer at Axon Active Vietnam

Artificial intelligence (AI) has been changing the way software is tested and how humans interact with technology. AI predicts, prevents and automates the entire process of testing using algorithms. It will not only support and improve the models and test cases but also provide more sophisticated and refined form of text recognition and better code generators. Using AI will help to save time for testing and ensure a better quality software.

  • Be the first to comment

[DevDay2019] How AI is changing the future of Software Testing? - By Vui Nguyen, Software Test Engineer at Axon Active Vietnam

  2. 2. 2 Vui Nguyen Software Test Engineer at Axon Active Viet Nam
  3. 3. Agenda 3 What is Software Testing? 1 What is AI? 2 What challenges can AI solve? 3 Advantages and Disadvantages of AI 4 AI Tools for Software Testing 5
  4. 4. What is Software Testing? International Software Testing Qualifications Board: “Software testing is a process of executing a program or application with intent of finding the software bugs. It can also be stated as process of validating and verifying that a software program or application or product meets the business and technical requirements that guided its design and development” 4
  5. 5. Software Testing Overview 5 Requirement Analysis Implement Test Script Write test cases Execute test cases Compare Test result == expected value Expected Result Test result != expected value Expected Results or properties
  6. 6. Software Testing Challenges 6 Years Months Months Weeks Weeks Days Days Minutes
  7. 7. Software Testing Challenges 7
  8. 8. What is AI? 8 It is the science and engineering of making intelligent machines, especially intelligent computers to understand human intelligence, but AI does not have to confine itself to methods that are biologically observable STANFORD “Artificial intelligence (AI) is a self improving enable horizontal layer that is solving problems that were in the realm of science fiction for the past several decades” Jeff Bezos, Inc Artificial intelligence (AI) makes it possible for machines to learn from experience, adjust to new inputs and perform human like tasks SAS Artificial intelligence is the boarder concept of machines being able to carry out tasks in a way that we would consider “smart” FORBES
  9. 9. Structure of AI 9
  10. 10. Applications of AI 10 Fraud Detection • AI has the ability to identify fraudulent behavior, as well as identify what next pattern of suspicious behavior will be • AI algorithms can preempt such fraudulent transactions and can lead to huge costs savings for banks and its customers Meeting regulatory requirements • AI is used to ensure that regulatory requirements are met and that data is kept with monitoring done on a real- time basis. This allows issues to be flagged a lot sooner Boost customer engagement • AI will assist in the creation of customized and intelligent products and services, with new features, more intuitive interactions (e.g. speech) and advisory skills (e.g. personal financial management) BANKING Computer aided diagnosis • AI is being used extensively to read and interpret complex radiology, pathology reports to help doctors arrive at early High risk groups identification • AI is being fed huge volumes of data historical medical records that helps in identifying whether a patients is in a high risk group for any particular disease say stroke cardiovascular diseases or cancer Epidemic out break prediction • ML and AI technologies are also being applied to monitoring and predicting epidemic outbreaks around the world, based on data collected from satellites, historical information on the web, real- time social media updates, and other sources • Support vector machines and artificial neural networks have been used, for example, to predict malaria outbreaks, talking into account data such as temperature, average monthly rainfall, total number of positive cases, and other data points HEALTHCARE 10
  11. 11. Applications of AI 11
  12. 12. Processes of software testing are similar to the processes used to train AI AI for Software Testing AI applications use the output of the AI training process and apply that to specific problems, such as recognizing a stop sign and stopping a car, traffic lights … which consist of inputs and comparing the outputs to expected results 12
  13. 13. AI Training data Process  Training AI systems is very similar to testing  AI training process assigned into categories: processing, sensing, learning 13
  14. 14. What challenges can AI solve? 14 Ease of Authoring and Executing Tests Releasing at the Speed of Development Reducing Maintenance and Eliminating Flaky Tests Faster and More Stable UI Tests Continuous Learning from Production Data Removing Dependencies
  15. 15. Hundreds of attributes used to identify elements  A few changes don’t break the test  Automatic get best location strategy to successfully identify 15 Faster and More Stable UI Tests
  16. 16.  AI’s self-healing mechanism can detect problems in the failed tests before they even occur, fix tests instead of us reacting to them  AI can figure out which tests are stable or flaky, analysis what tests need to be modified to ensure test runs are stable  Based on large numbers of test runs, AI can optimize the wait times used in tests to wait for the pages to load and also can handle tests running on different resolutions Reducing Maintenance and Eliminating Flaky Tests 16
  17. 17.  AI will start observing and learning how our customers are using the product and can start creating tests based on real user data  AI will identify commonly used actions such as logging in/out of the application and cluster them into reusable component 17 Continuous Learning from Production Data LEARN BY OBSERVATION (PRODUCTION) AGGREGATE USER ACTIONS INTO FLOWS TEST PRODUCED FROM FLOWS
  18. 18.  Once we have authored some tests and have run them consistently for a period of time, the AI can start recording all the server responses  When run the tests again, instead of talking to a server or database, the test will access the stored responses and will continue to run without any obstacles 18 Removing Dependencies
  19. 19. Author and execute tests can be done in a matter of hours Use dynamic locators and the ability to easily create reusable components Integrate CI/CD systems easily with public and private grids Nontechnical people can get involved in test Increase collaboration within teams and encourages everyone to own the test automation effort Ease of Authoring and Executing Tests 19
  20. 20.  With AI powering the transition to autonomous testing, reducing the maintenance to a minimum, and creating more reliable tests, the ability for teams to release faster is better than  Testers an maximize user coverage by connecting authoring of tests with production apps mapping to real user flows  We have the ability to take a risk-based approach and base our decisions on real data  We are now able to create more user scenarios in short period of time. This means you can find bugs fast and release faster 20 Releasing at the Speed of Development
  21. 21.  Improved accuracy and efficiency  Overcoming the limitations of manual testing  Benefits both testers and developers  Improving overall test coverage  Time & cost-saving Advantages 21 Advantages and Disadvantages of AI
  22. 22. Advantages and Disadvantages of AI  Artificial intelligence software testers use the concept of GIGO (Garbage in Garbage Out)  High costs  Can’t think outside the box Disadvantages 22
  23. 23. AI Tools Visual Testing Regression Test Monitoring / Report 23
  24. 24. Visual Testing 24 Baseline <empty> Image1 Image2 Image3 Result: Image1 Image2 Image3 new new new visualTest () { simulate UI state 1 check (“check 1”) simulate UI state 2 check (“check 2”) simulate UI state 3 check (“check 3”) } 1) Write app & test code 2) Run test 1st time 4) Baseline created3) Review results Baseline: Image1 Image2 Image3 visualTest () { simulate UI state 1 check (“check 1”) simulate UI state 2 check (“check 2”) simulate UI state 3 check (“check 3”) } Image1F Image2 Image3B Result: Image1 Image2 Image3 diff == diff 5) Update app & test code 6) Differences detected 8) Save baseline7) Review results Baseline: Image1F Image2 Image3 Baseline: Image1 Image2 Image3 3B: Bug 1F – New feature
  25. 25.  Segmentation algorithm, Convolutional Neural Networks and a combination of algorithms  Can be directly incorporated into testing frameworks  Test results available in Test manager 25 Visual Testing
  26. 26.  Flaky tests  Maintenance  Huge data  Learning curve 26 Regression Testing
  27. 27. Regression Test  Focuses on reducing flakiness  Data driven testing  Automated test case generation  Reads production user access 27
  28. 28. Monitoring / Reporting  Different tools, frameworks and test for functional, performance and security testing  Numerous test cases  Non prioritized test suite  Get meaningful data out of logs 28
  29. 29.  One tool for code coverage for different technologies  Prioritizes test cases  Functional, Performance, Security  Segregates test into smoke and regression  Failure prediction though log monitoring  Logs with reason for test failure 29 Monitoring / Reporting
  30. 30. 30 References  key-trends-watch  software-testing/   software-testing/  
  31. 31. Thank you for listening!