Sharing my experience how to build an automation solution based on Appium that provides a possibility to run same mobile automation tests on both iOS and Android platforms. Also, going to show how to integrate mobile automation into a main development process, will include useful tips & tricks for mobile automation on real devices and practical demonstrations with code sharing.
Building an advanced automation solution based on Appium
1. Building an Advanced Automation
Solution based on Appium
Anton Sirota
QA Automation Expert
DataArt
2. About me
QA Automation Architect/TeamLead
Participated in more than 30 projects
Leader of QA Automation Community
Certified Trainer and Lector at DataArt QA School
The author of Automation courses
С#
4. • Shift from QA to Quality Engineering
• IoT
• DevOps
• Big Data Testing
• Mobile Test Automation
• Open Source Tools adoption c
World Quality Report Summary
5.
6. QA Automation statistic DataArt
33%
24%
18%
14%
6%
5%
Web API Mobile Performance DataBase Desktop
7. Tools Comparison
Appium Espresso XCUITest Calabash Selendroid Robotium
iOS + - + + - -
Android + + - + + +
Script Language Almost
any
Java ObjectiveC
/Swift
Ruby,
.NET, Java
Almost Any Java
Emulator + + + + + +
Real device + + + + + +
Same test for
two platforms
+ - - - - -
Cost Free Free Free Free Free Recorder -
paid.
Community Very
Active
Google Apple Average Average Average
Speed Slow Fast Fast Medium Slow Medium
10. Find coordinates to click and send keys
Use tesseract tool to get needed text
For non-accessible elements
11. Bash scripts to reset all data before start
Top hardware (ex. latest mac mini with SSD)
Worse hardware - less parallel runs
Get better stability on emulators
12. Additional required actions
Open system settings
Start and stop and switch application
Change System Parameters (date, location, etc.)
Read and write files
Get system notifications
13. Use additional adb commands
Run Appium sever with -relaxedSecurity parameter
23. 1. Build Android Application
2. Upload build to cloud or file server
3. Run your tests always under the latest build
Android
24. 1. Get 3rd party (CocoaPods, Carthage).
2. Fastlane:
2.1. Establish code signing and provisioning.
2.2. Make the build.
2.3. Upload the build on cloud or file server
2.4. Upload the build to TestFlight
3. Run your tests under the latest build
iOS
26. Android Connection via wi-fi
N O T E : D E V I C E H A S T O B E I N T H E S A M E N E T W O R K A S Y O U R C O M P U T E R
26
• Connect your device via usb cable
• Get an ip address of your Android device
• Run in console: “adb tcpip 5555”
• Run in console: “adb connect your_phone_ip”
• Remove usb and run in console “adb devices” to make sure your
device is connected
27. N O T E : D E V I C E H A S T O B E I N T H E S A M E N E T W O R K A S Y O U R C O M P U T E R
27
iPhone connection via wi-fi
• Connect your device via usb cable
• Open your project in xcode
• Open devices list and find your device
• Click on the checkbox : "Connect via
network“
• Once done (*You'll see a globe icon in
front of device), remove the usb
29. Case Study
Application to scan QR Codes
Challenges
• Scanner supposed to be connected to a real device and use it’s only one port
• Smoke suite needs to be automated
Solution based on Appium
• Connect real android device using Wi-Fi and run tests as usual
29
30. Case Study
Call quality verification
Challenges
• Make a call from one phone to the other
• Pass some voice (audio file) from one phone and record it on the other side
Solution based on Appium
• Use two real devices: one that enters the phone number and the second that answers
• Use custom cable and VLC command line to record the audio from the second device
30
31. Tips and Tricks for iOS Real device
Certificates
To be able to run tests on iOS real device you need to have:
• Apple’s developer account
• Certificate installed on your Mac machine
• Provision profile for an application you want to test
• Device should be included into provision profile
31
33. Real devices
Advantages Disadvantages
33
Good tests run time
Easy to setup (Android)
Stable long time testing on the same
device
Available: GPS, Network, Bluetooth,
Camera, Gyroscope
The real devices are costly compared
to the emulator/simulators
Maintenance of real devices (OS
updates, reboots, etc.)
34. Emulators/Simulators
34
Advantages Disadvantages
Mostly free
Easy to setup
Easy to debug
Can run many configurations
Sometimes behavior can be different
on real device and Emulator
It is slower as compared to actual
devices
No possibility to do performance
testing
Not all functions are present
35. What’s my choice?
Real Device Testing = Real Results 35
Real Devices Emulators/Simulators
Should be used on the most projects
with Native/Hybrid applications.
Acceptable for Mobile Web
Applications testing
Acceptable during the PoC stage
Need to run under the many
configurations
38. Advantages of Cloud Services
38
• Don’t spend any time for environment setup and support
• Stable environment and test results
• On some cloud services tests run faster than locally
• Much easier and faster to setup multithreading
• You can use bunch of different configurations
• Mostly good support and help with problems solving