This document discusses the evolution of software testing and how artificial intelligence is being applied to improve testing. It provides examples of intelligent testing tools that use AI to reduce flaky tests, increase test coverage, prioritize test cases, and predict failures. Such tools apply techniques like machine learning algorithms, computer vision, and natural language processing to testing tasks. The document also compares several popular AI-based testing tools and their pricing structures.

1. Testing Tools With
AI
w e l c o m e
By,
Haritha Hari
Email: harithahari13@gmail.com
Twitter: @HarithaHari1

2. Evolution of software testing
• Adhoc testing
• Manual testing
• Commercialized testing
• Open source testing
• AI testing (dubbed third wave)

3. What is AI?
3
Intelligence demonstrated by machines, in contrast to the
natural intelligence displayed by humans and other
animals.
It has been around since the 1950’s

4. Why AI for software testing?
• Reduce Test flakiness => increases accuracy
• Increase test coverage =>High quality and
confidence
• Prioritize Test cases =>Less time to market
4

5. Artificial Intelligence for Software Testing (AIST)
5
The Artificial Intelligence for
Software Testing Association
(AISTA) is focused on applying and
extending AI to the world of
software quality in all forms and
obviating the need for human
testing activities.

6. Intelligent tools:
6
 1. Applitools
 2. Testim
 3. Sealights
 4. Test.AI
 5. Functionize
 6. Mabl
 7. ReTest
 8. Test Sigma
 9. ReportPortal
 10. Appvance

7. Visual testing tools
Complications
• Different browser versions
• Different monitors
• Font smoothing
• Graphic cards
7

8. AI Visual testing tools
8
• Visual Testing tool that does
visual verification with AI.
• More than pixel to pixel
comparison.
• Suitable for dynamic visual
content.

9. • Segmentation algorithm, Convolutional Neural Networks and a
combination of algorithms
• Available as mvn dependency, npm package, jar
• Can be directly incorporated into testing frameworks
• Test results available in Test manager
Tools:
9

10. Regression testing tool
Complications:
• Flaky tests
• Maintenance
• Huge data
• Learning curve
• RCA
1

11. AI Regression testing tool
Uses machine learning algorithms like MartiRank
• Focuses on reducing flakiness
• Data driven testing
• Automated Test case generation
• Reads Production user access
1

12. 1
Tools become smarter with more runs and increases the stability of test suites.

13. 1

14. AI Regression testing tool
Tools:
1
• Testim • Test.AI
• Mabl • Test Sigma
• Appvance • Retest
• Functionize

15. App monitoring / reporting tool
Complications
• Different tools, frameworks and tests for functional, performance
and security testing
• Numerous test cases
• Non prioritized test suite
• Get meaningful data out of logs
1

16. AI App monitoring / reporting tool
• One tool for code coverage for different technologies
• Prioritizes test cases
• Functional, Performance and security monitoring
in one tool
• Segregates tests into Smoke and regression
• Failure prediction through log monitoring
• Logs with reason for test failure
1

17. 1
Identifies code areas mostly used but not covered by any tests. Smartly identifies relevant
tests.

18. 1

19. Tool Comparison
19
Tools Use Price
Applitools Visual testing Licensed
Testim Reduce flaky tests
Free
Up to 100
executions/month
Sealights Identifies relevant tests and improves code coverage Licensed
Test.AI
Mobile regression tool and dynamically adjusts to locator
changes
Licensed
Functionize No scripting. Locator updation through screenshots
Licensed
Mabl Creates Smoke and Regression suites
Priced at the no:of runs
ReportPortal Predicts failures Open source
Retest Blacklisting and whitelisting Demo phase

20. References
• https://www.aitesting.org/
• https://dzone.com/articles/artificial-intelligence-in-testing-tools-and-advan
• http://test.ai/
• https://www.functionize.com/
• https://appvance.com/
• https://www.testim.io/
• https://applitools.com/blog/how-to-do-image-comparison-right
2

21. 2

22. Thanks
By,
Haritha Hari
Email: harithahari13@gmail.com
Twitter: @HarithaHari1