A big problem with test automation on any platform or operating system is synchronizing test automation interactions with the UI. It is challenging to know when the UI is ready for the next automated click(). Traditional black box tools try to address this problem by explicit or implicit waiting, but this technique is slow and error-prone. A new generation of test frameworks, starting with Espresso, understands the internals of the app and synchronizes interactions only when the view is ready, making the framework very fast and reliable. This same technique is making Cypress and EarlGrey popular. Join Satyajit Malugu to discover how peeking below the surface of the system under test makes tests less flaky. You'll leave with an understanding of synchronization, black box versus gray box testing, and how to implement them in your own frameworks.

1. MAKE YOUR UI
TESTS RESILIENT
WITH THE NEXT
GENERATION OF
FRAMEWORKS
BY SATYAJIT MALUGU
SENIOR ENGINEER, SOFI

2. SETTING CONTEXT
• About me: ~10 Automation experience primarily in mobile apps
• Intended audience: 1+ year experience with test automation

3. WRITING AUTOMATION IS EASY BUT
MAINTAINING IT GREEN IS HARD
• On paper UI automation is simple
• Find an element
• Interact with an element
• Assert state change
• Maintaining test passing reliably is very hard
• In a CI environment team can count on
• Over a period of time

4. BIGGEST PROBLEM WITH UI TEST
AUTOMATION

5. WHY IS IT FLAKY
• Classic E2E test have too many variables
• Network
• Data
• Test (lab) infrastructure
• Complicated application logic, 3rd party deps
• We don’t know for sure when SUT is ready
• Lack of control on SUT
System
Under Test

6. HOW DO YOU ADDRESS FLAKINESS
SLEEP WAIT PRAY

7. BUT ITS NOT ENOUGH
NOT RELIABLE MAKES AUTOMATION
SLOW
DIAGNOSING THE
PROBLEM TAKES LONGER

8. GREY BOX VS BLACK BOX

9. IS THIS BLASPHEMY?
•Can a puny test look inside the
sacred walls of the SUT?
•Of course, if that means more
reliable tests

10. SYNCHRONIZATI
ON
Bringing the SUT to a ready
state before performing any
testing operations on it

11. ESPRESSO
• Mature, de-facto Android test
automation framework from
Google
• Very fast and highly reliable
• Synchronizes the app with test
actions
• Network, UI loaders,
background tasks and custom
tasks can all be synchronized

12. HELLO WORLD ESPRESSO

13. ESPRESSO IDLING RESOURCES
An advanced framework support for async operations
Provides fine grained control for waiting
Makes tests much more robust

14. DEMO

15. CYPRESS.IO
• Non-selenium web automation
tool
• Runs within the JavaScript
engine of the browser
• All in one JS tool with extensive
tools for debugging like
timetravel

16. CYPRESS
DEMO

17. EARLGREY
• Very similar to espresso but for
iOS
• Synchronization at UI level,
network, and in custom events
• Can live side by side with default
XCUITests

18. EARLGREY
DEMO

19. WHAT'S THE TRADEOFF
Fidelity
Reliability Unit
tests Espresso,
Earlgrey
Cypress
Selenium,
Appium

20. TEST AUTOMATION PYRAMID
Black box
Tests
Grey Box UI
tests
API/Integration tests
Unit tests

21. TAKEAWAY
S
Think beyond selenium and appium
Consider adding grey box tools to
your test automation toolbox
Synchronization

22. QUESTIONS