Slides by Anton Hristov, Product Manager of mabl.
Watch the accompanying webinar: https://www.mabl.com/blog/end-to-end-automation-at-scale
Testing end-to-end user scenarios is challenging, yet more important than ever due to increased complexity, variety and importance of user interfaces. Delivering a quality user experience requires taking a holistic view of the end-to-end user journey, which can span across applications, browsers, devices and different modes of interaction such as touch and voice.
In this webinar, we will explore different ways mabl can help you create intelligent end-to-end tests that focus on the user journey and run at scale across browsers. No scripting necessary.
Learning outcomes:
Why we need to shift from quality assurance to quality intelligence
How to create intelligent tests quickly to increase coverage
What diagnostics information is available for root-cause analysis
When and how to reuse a set of steps across multiple tests
When and how to apply data-driven (parameterized) testing approach
4. Continuous integration and deployment (CI/CD) tools enable developers
to ship code much more frequently, often multiple times a day
software delivery has changed...
...but traditional testing tools haven’t changed much. 4
5. Outsourced QA
- Waiting for test results
takes too much time
- Long lead times on test
creation and updates
- Limited by legacy and
open source tools
Open/custom source
- Flaky, brittle and hard to
maintain tests
- Little diagnostics data, if any
- Highly specialized technical
skills, hard to onboard new
team members
5
Legacy tools
- Complex setup
and maintenance
- Costly infrastructure
- Limited DevOps workflow
support
Challenges with traditional testing tools and processes.
6. Waterfall → Agile → DevTestOps
1990s 2000s Today
DevTestOps is here..
Continuous delivery creates the need for continuous testing, and that
requires quality intelligence to operate effectively at scale.
6
8. 8
Dev Unit Integration End-to-end Production
Catch bugs sooner, before users do!
…and bring customer intelligence closer to the development team.
User
Shorten the feedback loop and increase the velocity of delivering value to users.
We love shift-left in testing….
10. Intelligent testing principles
1. Easy and fun to create, and auto-create
2. Adapt seamlessly to change
3. Run in the cloud
4. Produce insightful output
5. Part of continuous delivery pipeline (DevTestOps)
6. Support collaborative workflow
7. New definition of quality
14. User flows
● A flow is a common set of steps across multiple user journeys
● Typical testing use cases
○ Navigation to a page from different locations within the app under test
○ Login with different users - client portal vs admin portal
○ Application entity creation - user, product, ticket, etc.
○ Component testing - carousels, tables, etc.
● Often tested with variables and data-driven testing with parameters
21. + Capture screenshots and
detected visual changes
without any scripting or
configuration.
+ Dynamic content regions are
automatically detected to
reduce noise and help you
focus on what matters
Built-in visual testing and monitoring.
2121
22. Identify application load time issues.
+ mabl detected that
application performance
improved after a code
deployment
22
+ Speed index measures
perceived load speeds, similar
to Google Lighthouse tools
24. Data-driven testing
A software testing methodology where you repeatedly execute the same set
of steps using a data source to supply inputs and expected outputs for
verification.
29. Data-driven testing benefits
● Separation of concerns
○ functionality vs data inputs (parameters)
● Increased test coverage
● Eliminated redundancy
● Easier maintenance
● Expensive to manually test but cheap when automated
○ Takes advantage of parallel test execution
● Easy to add new testing scenarios as the application evolves
31. mabl DataTables summary
● Parameters are first specified in the mabl Trainer
● Test inputs and expected outputs are retrieved from a DataTable
● Support for copy + paste from a spreadsheet such as Microsoft Excel
● Tight association between a journey and a DataTable
● Each DataTable scenario triggers a separate journey run, in parallel
● Ability to overwrite Journey and DataTable association from plan settings
32. And one more thing… Security
● Encryption of all data in transit and at rest with a key for each customer
● Data isolation from other customers
● Secure tunneling to test internal/local environments
● Proactive approach to security
● Compliance with data privacy standards - GDPR, US/EU Privacy Shield
● Trusted partner