Traditional testing isn't keeping pace with the speed of DevOps and Continuous Delivery, but hope is not lost. AI will be used to make testing faster and more accessible to all. Existing QA solutions were built for a world where software changed infrequently. Highly adopted tools such as Selenium, Appium, and JUnit require a specialized skillset, and require too much maintenance once you start factoring in the brittle nature of tests, and the infrastructure required to run tests at scale. But there is still hope for QA in machine intelligence. Next-generation ML tools are here to help QA keep up with the agility of modern software delivery practices. You'll learn what ML strategies are being employed to make this possible, and what the future of QA - intelligent testing - looks like. Takeaways: - An overview of the QA tooling and vendor landscape - Demystifying the buzzwords - what machine intelligence in testing means today and tomorrow - How implementing DevTestOps can help increase test coverage and improve product quality

1. The Era of Intelligent Testing:
How Machine Intelligence is
Shaping the Future of QA
1

2. It’s nice to meet you!
Brian A. Brady
Enterprise Solutions Advisor, mabl
www.mabl.com
brian@mabl.com

3. ■ Driving forces of DevTestOps
■ Recent progress of AI and cloud in testing
■ How mabl supports this new era of “intelligent” testing
Agenda

4. 4

5. We are serving more diverse clients

6. Applications are more dynamic than ever
Credentials
Slow loads
Dynamic data
Dynamic
presentation
Dynamic
locators

7. DevOps makes effective
automation a top priority
7

8. mabl UI: Up to 100 commits per week
Commits
Week

9. “Development cycles are getting shorter
and new features are coming faster than
ever…There simply isn’t enough time to
test”
- Fortune 250 QA Leader

10. Outdated testing tools and processes
can’t keep up
Testing
Development Ops

11. 11
“Gartner anticipates that this research will
be the final Magic Quadrant for Software
Test Automation as it no longer provides the
correct perspective for understanding and
evaluating the vendor landscape.”
Source: Magic Quadrant for Software Test Automation, November 2019

12. Waterfall → Agile → DevOps → DevTestOps
1990s 2000s 2010s 2020s
The way we test software has
fundamentally changed

13. DevTestOps is here
Continuous testing is now part of software delivery

14. Cloud is one part of
the solution
14

15. Cloud enables bursty concurrency
Concurrency needed over the course of a day - by test window
1,500
15,000
100,000

16. Tests run in the cloud
■ On-demand containers/VMs
■ No resource bottlenecks
■ Flexible scheduling
■ No operational overhead
■ Data, data, data

17. Tools are becoming
more intelligent
17

18. “Applying AI and machine learning to QA...
is becoming essential to have an integrated,
intelligent and automated approach to
continuous testing.
Digital transformation at scale will be
[otherwise] impossible to achieve.
Source: Magic Quadrant for Software Test Automation, November 2019

19. Build smarter pipelines
Insight
Visibility
Automation
Intelligence

20. Visibility

21. Insight

22. Intelligence

23. The rise of Machine Intelligence Automation

24. The Era of Intelligent Testing Emerges
✔ Tests run in the cloud
✔ Tests are easy to create
✔ Tests adapt seamlessly to change
✔ Tests produce insightful output
✔ Testing solutions are part of CI/CD pipeline

25. Tests are easy to create
■ “Recorder” UX
■ Fallback heuristics
■ Continuous updates
■ No scripting experience needed
...but can write some code

26. 85%
Reduction in automated test creation time

27. Auto-healing reduces maintenance effort
Collected Attributes:
● ID
● XPath
● Innertext
● URL
● CSS class
● Custom attributes
● Parents / children /
siblings
● +more

28. Tests produce insightful output
28

29. DOM Snapshots

30. http archives

31. Chrome
traces

32. Visual change detection

33. Performance analysis

34. True end-to-end testing
1. 2. 3.

35. 50%
Increase in Deployment Speed

36. What’s next for Intelligent Testing?
36

37. “We have 15,000 tests.
Why are there still defects
in production?”

38. Understand & improve real user
journey coverage
38

39. If only there was a tool that tells
you what users were really doing….

40. Improving coverage

41. Baking functional UI testing into
the development workflow
41

42. Code
branch
QA ProdGitHub PR
Local server
app-local.mabl.com
app-dev.mabl.com app.mabl.com
feature_branch
master_branch
Traditional QA
Approach
Shift Right
Shift Left

43. Code
branch
QA ProdGitHub PR
Local server
app-local.mabl.com
app-dev.mabl.com app.mabl.com
feature_branch
master_branch
Preview
(ephemeral)
shift-left
Pull Request Support

44. GitHub Actions Integration

45. Bitbucket Pipes Integration

46. Code
branch
QA ProdGitHub PR
Local server
app-local.mabl.com
app-dev.mabl.com app.mabl.com
feature_branch
master_branch
Preview
(ephemeral)
shift left some more
Local dev workspace

47. Run mabl
tests via
CLI

48. Run tests locally for faster feedback in dev
■ Fast test runs in the background
■ Quick dev-friendly test creation
▲ CLI + Trainer
▲ Tests as code

49. 49
Participating in DevTestOps

50. “We always overestimate the
change that will occur in the
next two years and
underestimate the change that
will occur in the next ten.
Don't let yourself be lulled
into inaction”

51. ■ How do we shift quality left and right?
▲ What are the new job descriptions for testers?
▲ What are the new job descriptions for developers?
▲ How do we develop these new skills?
■ What roles will Selenium and Appium play in the new Era?
■ Where do we land on integration vs. best of breed tools?
■ How deterministic and independent are tests?
We have to talk

52. mabl.com/qa-highway