Rock solid UI Test Droidcon Berlin 2021
Oct. 25, 2021•0 likes•209 views
Report
Engineering
We Android developers use the "flakiness" word more than the "stability" word while talking about Espresso and Instrumentation tests. We cannot truly trust flaky tests. Instrumentation testing will continue to be part of our lives even with Jetpack Compose therefore, we need stability. Running all UI tests before merging any code is possible, and it saves us a lot with 400+ integration tests. Unfortunately, stability is not something that comes out of the box. A certain level of architecture, proper thread usage, and test setup with helper rules are required. There is an uncountable number of things that affect UI tests. Good thing, there are explanations and solutions for most of the flakiness. In this talk, you will learn more about best practices to increase test stability, how to utilise Test-Driven Development with UI testing, and how this will help you find bugs earlier. Sinan Kozak
Sinan KOZAKFollow
Recommended
Jessebrackenburymasccc2014Jen Boudrie
Microsoft SurfaceVivek Roodagi
Addressing Site Quality | Search Marketing Summit AustraliaBuiltvisible
PPC Cost Analysis | Search Marketing Summit Australia 2Builtvisible
SEO for Faceted Navigation | Get STAT City CrawlBuiltvisible
Benchmark Big Data Analytics (english)Stratebi
More Related Content
What's hot
Solita HUB: Product information in the heart of customer experience - Per-Ola...Solita Oy
Preliminary Task EvaluationLordMaxi8
Improve remediation success and reduce portfolio risk using a Lean approachHaley & Aldrich
Build a Strong Career Foundation by following Milad Yousif Hardworking wayMilad Yousif
Guidelines for designing effective english language teaching materialsTania Beltran Garcia
Audience magnetisation 101Tyron Love
Recently uploaded
Tha Statistics of future: Less Math and more Visual Statistical ThinkingLourdes Pozueta Fernández
GDSC PU Cloud Study Jam Intro. Session 23-24.pdfPOORNIMA UNIVERSITY
North American YAT-28 Turbo Trojan.pdfTahirSadikovi
Unit 4 Food packaging materials and testing.pptxDr P DINESHKUMAR
Airbus A319 Aircraft Airport & Maintenance Planning Manual.pdfTahirSadikovi
INDIAN KNOWLEDGE SYSTEM PPT chp-1.pptxRuchiSharma494176
Rock solid UI Test Droidcon Berlin 2021
Editor's Notes
- Every one is frustrated byy flaky tests
- Who stop using android test because off flakiness
- 800 tests run on every commit under 20m PR wont merge without succesful AndroidTest 18 Different features has their Android test in their module We dont depend on UI test. Only most important features are tests end to end Almost all main usacases covered
- User interface (UI) testing lets you ensure that your app meets its functional requirements and achieves a high standard of quality such that it is more likely to be successfully adopted by users.
- One approach to UI testing is to simply have a human tester perform a set of user operations on the target app and verify that it is behaving correctly Time-consuming, tedious, and error-prone
- The automated approach allows you to run your tests quickly and reliably in a repeatable manner. androidTest folder Creates apk to run on device
- UI interaction End to End flow of feature Device resources
- Pure java kotlin classes can be tested on jvm They are faster We should aim to have business logics in class we can test
- In reality we wont be able to test everything on jvm User interface Device resource usage End to end verification from user persfective
- Who stop using android test because off flakiness
- There is no out of box solution Device control. We want to focus on WHAT to test Delegate device control as much as possible Threading issues Framework issues
- APIs to write AndroidTests. Dont uses your classes Use app from user persfective
- Espresso focuses on how to find and how check views Espresso is aware of multiple things
- Espresso is low APIs focus on HOW to test
- As developer we want to focus on WHAT to test Espresso is API around actions and espresso is has HOW to find views or HOW to check conditions
- 10 minutes
- How we can reduce learning curve Kakao is a library around Espresso APIs.
- The goal of the Kakao is to provide better readability and discoverability for Espresso testing. Kakao doesn’t aim to change the behaviour of Espresso. By default, Kakao enforces the same mentality of robot pattern. Kakao expects a Screen class. And in Screen class, you can define common UI elements or actions using Kotlin DSL.
- Kakao has solutions for common UI element Easy to discover Type safety about actions and check Easy to read tests But this is not about stability
- Lets talk about most common usecases 1st animation
- Because Espresso isn't aware of any other asynchronous operations, including those running on a background thread, Espresso can't provide its synchronization guarantees in those situations. In order to make Espresso aware of your app's long-running operations, you must register each one as an idling resource.
- For all other background operations, test authors should register the custom resource via IdlingRegistry and Espresso will wait for the resource to become idle prior to executing a view operation. It would be great if we can have one idling resource for threading solution we have. And actually we have some
- 20 minutes
- For all other background operations, test authors should register the custom resource via IdlingRegistry and Espresso will wait for the resource to become idle prior to executing a view operation.
- For all other background operations, test authors should register the custom resource via IdlingRegistry and Espresso will wait for the resource to become idle prior to executing a view operation.
- There is not a clean solution We can use similar provider solution as ExecutorProvider It is not possible to prevent libraries to use Default dispatcher SetDefault vs setMain With rx all of background operations covered It feels like we are not goiing forward but keep craeting new problems
- If you remember the usecases for espresso maybe we can find a different solution. Robost solution
- If you remember the usecases for espresso maybe we can find a different solution. Robost solution
- If we cannot control concurency framework, we can track background operations directly
- For all other background operations, test authors should register the custom resource via IdlingRegistry and Espresso will wait for the resource to become idle prior to executing a view operation. Create an interceptor and register idling resources with the information you have. In interceptor you can track current network operations.
- After the network operation, you set the data to live data. Everything will be good, right? No
- We set a countiing executor for arch component. DataBinding will use value of livedata and there wont be a flakiness, right? No
- There is no easy way to keep track of when data binding applies changes to screen. There is only hasPendingIntent information Simple solution is not using databindiing But if yyou have databindiing, yyou need to regiister and idliing resource and check all the views. All theeee views
- Network, livedata and databinding wont be a problem. We can use ListAdapter and submit a new list and everything will allright? You probably already understand the theme of this flow. No
- We started a data flow network layer until data reaches to screen, we needed 4 solution. We will definitely need moree solutions That is why it is nice to have a single place to share solutions 30 minutes
- There are different options to run UI tests. You can create your own test farm in the company with real devices. You can use Github actions and run an emulator on Github action. You can use other build frameworks like Jenkins and set up your own emulator on CI. All of those options will need maintenance and they will be bound to your investment of money and time.
- Run on google infrastructure All of the devices on Firebase Test Lab have special adjustments We prefer to use virtual devices as there is generally no queue for them They are also active in Firebase slack. You can ask questions or report issues. And from my own experience, they help individual cases.
- One cool feature of Firebase test lab is sharding Split tests into buckets and run them on different devices to parallelize execution Cost will be same Duration will depend on how small the test buckets are
- Creating test matrices that should finish under desired time is a difficult problem. Luckily we don’t do that setup by hand.
- We need gradle integration for Flank And Fladle is gradle wrapper
- We have pipeline Cover more usecases at scout Good improvements coming to android testing
- Espresso is great library. I always feel that espresso is just starting point like any other coffees. They provided base element so we can add milk or water and use it