There's a shift to open-source mobile test automation tools happening today among developers and QA. Many mature technology sectors are adopting lightweight, vendor-transparent tools to fulfill the need for speed and integration. But as with many free and open-source software markets however, a plethora of tools complicates the selection process. How do you know which tool is worth spending the time to learn and integrate in your own environment? These slides aim to help you choose which open-source test automation framework to use based on a number of critical considerations.

1. Choosing the Best
Open Source Test
Automation Tool for You

2. So you’re trying to decide which
open source test automation tool
to use…

3. Which Tool is Best?
7/27/2016 3© 2016, Perfecto Mobile Ltd. All Rights Reserved.
SELENIUM APPIUM CALABASH ESPRESSO XCTEST UI
When looking at today’s open-source mobile test automation landscape, there are five
highly-adopted test frameworks.
We have summarized each approach to help you decide which makes the most sense for you.
Each tool has advantages for your mobile and web testing depending on your needs.
While reading, consider which of the functions
you need most for your own test automation purposes.

4. Making the Decision
A Comparison
Click the table for a more
detailed analysis of each
test automation tool:
The next slides offer a
breakdown of each tool

5. Selenium
• The best choice for web test automation
teams testing for responsive web design
or stand-alone web sites
• Less suitable for developing unit testing,
making this framework less appealing for
developers
• Its core test reports are not highly
informative and lack unique mobile related
insights

6. Appium
• Best suited for QA teams testing the
functionality of native, mobile web and
mobile hybrid apps across iOS and Android
• Less suitable for developers who wish to
develop and perform unit testing
• Appium reports are a bit limited from a
debugging and fast feedback loop
perspective, and do not include videos,
network logs and key vitals information

7. Calaba.sh
• Designed for organizations that work in
behavior-driven development workflows
• Offers an easy path to both develop and
test features in parallel, all in an easy
user-flow based language
• Appealing for both dev and QA
practitioners
• Provides solid insights and reports to both
dev and QA teams

8. Expresso & XCTest UI
7/27/2016 8© 2016, Perfecto Mobile Ltd. All Rights Reserved.
ESPRESSO XCTEST UI
• Both are designed for target users
• Espresso is for Android and XCTestUI is for iOS
• Both tools are fully integrated into development IDEs such as Android
Studio/Xcode, and offer very easy-to-develop techniques, including
test recorders
• Fully maintained by Google and Apple, which assures they always
support the latest OS features so developers can stay ahead of the
market and test accordingly
• Support both unit testing types and functional UI testing
• Both are app-context only, which limits their abilities to test for user
condition scenarios

9. Things to Consider
• What are your application use cases? What level of complexity
needs to be tested?
• Heavy UI elements?
• Environment dependencies (Networks, GPS, camera)?
• What OS versions and API levels should be supported?
• Does the app support multiple platforms? Mobile (iOS/Android)?
Web?
• Would your apps benefit from user condition testing?
When choosing a test automation tool
If you heed these considerations, you’re more likely to see long-
term success with your automated testing than if you dive into a
framework without understanding the implications.

10. Making the Decision
A Comparison
Review this table again to
locate the tool that offers
the capabilities you need
the most.

11. For more information about
automation test frameworks,
visit perfectomobile.com