Learn about challenges and how to improve your functional tests for fast and stable execution on real devices in a cloud like Sauce Labs.

1. Continuous mobile automation
with Appium in real device
cloud
MoT Berlin @ SauceLabs

2. hello!
I am Dmitry
Sr. QA Automation Engineer @
2
automateMeNow dmle

3. It’s not about:
✘ Appium vs Espresso / XCUITest
✘ Real devices vs emulators
✘ Sauce Labs vs other cloud providers
3
But:
✘ Get familiar with potential challenges
✘ Minimize test flakiness and get the most out of Appium
✘ Stabilize test execution in real device cloud

4. What works for me…
4
May not work for everyone

5. 1.
Let’s start with basics
2 + 2 = 4
5

6. Keep in mind while using Appium
✘ Install app only once
6
✘ Stop using bad locators
✘ Implicit wait = sleep = bad practice
✘ Every appium call counts
✘ Take care of keyboard
✘ Use API to generate test data / change app status
✘ Start activity / relaunch app / deep links between tests

7. Fast tests = Avoid using XPath
Fast & reliable locators
✘ id
✘ Accessibility id
✘ Predicate string
7

8. “- Don’t you ever use XPath?
8
XPath of total locators (~200):
Android: 6%
iOS: 4%
- I do. Only when really necessary

9. 9
Starting using device cloud like...
Source: http://en.espn.co.uk/f1/motorsport/image/6007.html

10. 10
After 3-4 sessions...
Source: https://insurance4motorsport.blogspot.com/2013/08/formula-one-driver-insurance.html

11. 2.
Local vs cloud
11

12. Be careful with version you use
12
appium-base-driver@3.12.0
appium-xcuitest-driver@2.102.0
appium-xcode@3.7.2
WebDriverAgent
appium-ios-driver@2.13.0
… 40+ modules
java-client@6.1.0
python-client@0.31
ruby-lib@9.16.0
wd@1.11.0
webdriverio@4.14.0
Server Client

13. Cloud is black box
13
appium-base-driver@3.12.0
appium-xcuitest-driver@2.102.0
appium-xcode@3.7.2
WebDriverAgent
appium-ios-driver@2.13.0
… 40+ modules
java-client@6.1.0
python-client@0.31
ruby-lib@9.16.0
wd@1.11.0
webdriverio@4.14.0
Server Client
In Cloud only release version:
1.9.1
1.8.1
1.8.0
1.7.2

14. 3.
Device selection
14

15. Devices have different:
15
✘ OS version
✘ Manufacture customized OS
✘ DPI / Screen resolution
✘ CPU / RAM
✘ Default connectivity type
✘ Keyboard
✘ Preinstalled apps

16. findElementById(“highlights”)
Samsung Galaxy S9
Motorola Moto G4
16Source: https://play.google.com/store/apps/details?id=in.olx.cashmycar.dealer

17. “app_india”: [
“Huawei_Mate_Pro_real”,
“HUAWEI_P20_real”,
“Samsung_Galaxy_S9_real”,
“Samsung_Galaxy_S8_Plus_real”
“LG_G7_Thinq_real”,
“OnePlus_6_real”,
“ASUS_Zenfone_5_real”
]
Use device matrix
17
✘ Based on users / local market stats
✘ Different manufactures / OS versions
1. Stabilize tests for matrix
2. Remove unreliable devices
3. Sync matrix every 3+ months
4. Keep it carefully from growth

18. Define device for test run
API:
/available
18
Filter
matrix
Select
random
Start
session
Dynamic allocation
filter by OS version & device type
static

19. UI Automator
Search for hidden elements inside scrollable View
findElementByUIAutomator(“new UiScrollable(new UiSelector()...”)
19

20. 4.
Connectivity Type
20

21. Check connection on session start
Keep in mind:
✘ Airplane Mode: 6+
✘ Data connection: <=4.4
✘ Wi-Fi connection: all versions
21
✘ getNetworkConnection()
✘ setNetworkConnection()

22. Network Connection (Android only)
22
Value Data Wifi Airplane Mode
0 (None) 0 0 0
1 (Airplane) 0 0 1
2 (Wifi only) 0 1 0
4 (Data on) 1 0 0
6 (All on) 1 1 0

23. 5.
Localisation
23

24. Check language on session start
✘ Check the element text on login
✘ Change via switch button
In worst case
✘ Skip tests with localisation checks
24

25. 6.
Device allocation
25

26. Once in a while getting exception
UnknownError - An unknown
server-side error occurred
while processing the
command. Selenium error: Lost
contact with app, please check
Appium log for more
information
The environment you
requested was unavailable.
Unable to install app to the
device XXX.
Unable to allocate device YYY
within 900000ms or 3 tries.
This can happen if there were
no available devices within the
time or your plan's max
concurrent test run is not
corresponding to your usage
26

27. 27
Smart device allocation
Source: http://en.espn.co.uk/f1/motorsport/image/6007.html

28. 3 iterations of custom mechanism
Version 1
Pull device matrix
Get available devices via API
Filter matrix with available
Select random one for session
Dynamic allocation
Version 2
Run v1 for EU device cluster
Run v1 for US device cluster
Dynamic allocation
Version 3
Run v2
Catch session creation error
Add device to “black list”
Retry v2 with black list filter
28

29. Device allocation handling
Select
Device
29
Allocate
Device
Done
Failed
Add device to Blacklist.
Retry

30. 7.
Appium session
30

31. “- Run each test separately
31
- It takes too long
- Use device caching
- Still too long

32. Session start ~ 40 sec
IOS ~ 70 sec
Running suite of 30 tests
32
Cached restart ~ 18 sec
IOS ~ 25 sec

33. 33
Session restart timeout
Source: https://www.mclaren.com/formula1/heritage/bruce-mclaren/

34. Custom sesion restart timeout
1. Define own session restart timeout (5 - 7 min)
2. Start session with time counter and enabled device caching
3. Starting from 2nd test check timeout - counter > max_test_time
4. Restart session before N test when timeout reached
5. Save result in SL, get report link, continue execution
34

35. Appium session handling
New
session
35
Close current session & start new with cached device
Test 1
Check
session
time
Test N
Retry

36. 30 tests ~ 13 min
Session timeout of 7 min, 1 restart
39%Time saved!
28 restarts ~ 8.4 min
18 sec / restart
36

37. Useful resources:
37
✘ http://appium.io/docs/en/about-appium/intro
✘ https://appiumpro.com
✘ https://wiki.saucelabs.com/display/DOCS
✘ https://stackoverflow.com/search?q=appium

38. Make Appium a better tool
Follow the progress:
Image comparison:
https://github.com/appium/appium/blob/
master/docs/en/writing-running-appium/
image-comparison.md
Element find plugins:
https://github.com/appium/appium/blob/
master/docs/en/advanced-concepts/ele
ment-finding-plugins.md
Contribute:
✘ Create issues
✘ Add/translate docs
✘ Answer questions
✘ Add new functionality
✘ Support client libraries
38

39. Let’s summarize:
39
✘ Follow the best practices with Appium
✘ Don’t try to support tests for all the devices in cloud
✘ Make sure to set correct connection and language on device
✘ Don’t fully rely on cloud with device allocation
✘ Be critical to test time execution, try to improve

40. 40
Back on track!
Source: https://www.juniqe.com/formula-1-poster-alu-frame-3445779.html

41. thanks!
Any questions?
You can find me at
41
automateMeNow dmle