The document discusses the role of artificial intelligence (AI) and machine learning (ML) in automation, explaining how AI can learn from human behavior and improve processes like test automation. It highlights various AI automation tools, such as Applitools, Mabl, and Testim, which aim to enhance efficiency by reducing maintenance time and adapting to changes in applications. Additionally, it provides insights on machine learning algorithms and their application in predictive modeling, emphasizing the need to experiment with different algorithms for optimal results.

1. AI For Automation
Presenter: Trung Lu
Senior Test Automation at KMS
Technology

2. Agenda
Introduction1
2
3
4
5
6
What Is Artificial Intelligence/ Machine Learning?
Machine Learning Algorithms
AI In Automation
Demo
Q&A

3. 2/ What is Artificial Intelligence?

4. What is Artificial Intelligence?

5. 2/ What is Artificial Intelligence?
Answer:
The power of a machine
to copy and learn from
intelligent human
behavior

6. Artificial Intelligence VS. Robot
AI
Programmed to think
Social Interaction
Learns
Robot
Programmed to do
Low level interaction
Only as smart as program
VS

7. How can?

8. AI Techniques

9. 3/ What is machine learning?
• Machine Learning at its most basic is the practice of using
algorithms to parse data, learn from it, and then make a
determination or prediction about something in the world. So
rather than hand-coding software routines with a specific set
of instructions to accomplish a particular task, the machine is
“trained” using large amounts of data and algorithms that give
it the ability to learn how to perform the tasks
• With machine learning, there is no exactness. There’s a range
of valid possibilities that could occur based on what the
machine has learned

10. Machine Learning Types

11. Supervised Algorithms
• A common principle that underlies all supervised machine learning
algorithms for predictive modeling:
Machine learning algorithms are described as learning a target
function (f) that best maps input variables (X) to an output variable
(Y): Y = f(X)
• This is a general learning task where we would like to make predictions in
the future (Y) given new examples of input variables (X). We don’t know
what the function (f) looks like or its form. If we did, we would use it directly
and we would not need to learn it from data using machine learning
algorithms.
• Sumary:
As a result, you should try many different algorithms for your
problem, while using a hold-out “test set” of data to evaluate
performance and select the winner.
The algorithms we try must be appropriate for our problem, which is
where picking the right machine learning task comes in.

12. Some example uses AI in actual
• Siri application in Apple
• Google Now
• Cortana
• …

13. 4/ AI In Automation

14. How AI Effects Automation?
DATA CREATION MAINTENANCE

15. Data Creation
Advertising application

16. Maintenance
• One of the common problems that
most of the automation testers face
is to re-work/ maintain the
automation test suite whenever the
applications go through a change.
• > 30% of testers time is spent on
maintenance

17. AI Automation Tools
AI
Automation
Tools
Mabl
Retest
Sealights
Applitools
Testim
Test.AI

18. AI Automation Tools
Applitools
1. Visual testing tool.
2. Leveraging ML/AI-based for
automated maintenance
(being able to group together
similar groups of changes
from different
pages/browsers/devices).
3. Being able to automatically
understand which changes
are more likely to be bugs vs.
desired changes and
prioritize diffs.
Testim
1. Testim tries to leverage
machine learning to
speed up the authoring,
execution and most
importantly the
maintenance of
automated tests.
2. Their goal is to help you
to start trusting your
tests.
3. Testim focuses on
reducing your flaky tests
and test maintenance,
which they see as one of
the most significant
challenges for most
organizations.

19. AI Automation Tools
Mabl
1. Eliminates flaky tests–like the other
AI-based test automation tools, Mabl
can automatically detect whether
elements of your application have
changed, and dynamically updates
the tests to compensate for those
changes.
2. Mabl can continuously compare test
results to test history to quickly
detect changes and regressions,
resulting in more stable releases.
3. Mabl helps identify and surface
problems quickly, alerting you to
possible impacts before they impact
your customers.
Retest
1. Use an artificially intelligent
monkey to fully automatically
test your application. That’s
how ReTest markets itself.
2. ReTest claims to be different
from other test automation
tools because it was built
specifically with testers in
mind.
3. It also stems from an artificial
intelligence research project,
so it tries to bake that AI
intelligence into their tool,
effectively eliminating the
need for their users to possess
any programming skills.

20. AI Automation Tools
Test.AI
1. Test.AI is billed as a tool that will add
an AI brain to Selenium and Appium.
2. The AI identifies screens and
elements dynamically in any app and
automatically drives your application
to execute test cases.
3. It’s smart enough to know that if an
element ever changes it can adjust
and identify it with you having to
make any manual changes. This tool
is still in beta.
Sealights
1. Sealights is a Cloud-based
platform. We all know that
developers and QA–both
managers and engineers–
are super busy these days
using CI and CD practices,
where they have frequent
releases and not enough
time to test the entire
application multiple times.
2. That’s one of the main
reasons Sealights was
created.
3. With their machine
learning-like technology
that analyzes both your
code and the tests that run
against it, it lets you know
exactly what your tests are
covering and what they’re
not.

21. AI Automation Tools
Sauce Labs
Sauce Labs ensures your favorite
mobile apps and websites work
flawlessly on every browser,
operating system, and device

22. 5/ Demo

23. How the Applitools works
We just "checkWindow", the
screenshot is uploaded to a
server, and that server uses AI
techniques to validate the
screenshot.

24. How AI techniques applied in Applitools
• Applitools is just looking at the differences between the actual page and the
expected (baseline) and you see if all the pages include the same change, the
Applitools will understand that it is the same page and show it to the human as
one change.
• Moreover, the tool will look at the layout of the page and the content of the
page, and categorize each change as a content change or a layout change.
• This will help us if we want to test responsive web sites—even if the layout
changes slightly, the content should be the same.

26. 6/ Q&A

27. Change is the law of life. And those
who look only to the past or present
are certain to miss the future.
-- John F. Kennedy
THANK YOU