The document provides an overview of mobile app testing methodologies and tools, highlighting the expertise of Danny Preussler in Android engineering and testing frameworks. It discusses various testing strategies such as instrumentation tests, UIAutomator, Robotium, and their applications along with performance metrics for mobile apps. The content also emphasizes the importance of speed, reliability, and scalability in testing, while mentioning popular open-source tools used by industry leaders.

1. Danny Preussler | Groupon
Rockstar Android Testing

2. about me
• Android Engineer @ Groupon
• Lead Engineer Android @ ebay Kleinanzeigen
• Team Lead Mobile @ Cortado
• C++ Developer @ PSI AG
• Started career as developer @ Alcatel
• Articles in: android360, mobile Technology, Java Magazin
• Speaker: DroidCon.de/nl, CONFESS, Blackberry DevCon
Believer in Testing

3. Sources: Internal Data; iTunes ranking for US stores available here - https://itunes.apple.com/WebObjects/
MZStore.woa/wa/viewFeature?id=500873243&mt=8&v0=www-itunes25Bcountdown-appstore
Nearly 70 million people worldwide
have downloaded our mobile app to
date; 9 million in Q4 2013.
One of the 25 most downloaded
free apps of all time
Nearly 50% of our global transactions
completed on a mobile device in
December 2013
a mobile company
Our mobile app is available in 43 countries

4. Android Testing?
Android Firewall by Uncalno, CC BY 2.0 flickr.com/photos/uncalno/8538679708
Robotium
Appium
Calabash
Robolectric
Espresso
Cloud
Crowd
Real Device?
uiautomator
monkeyrunner
Emulator
Selenium?
InstrumentationTest

5. Rockstar Testing?
• Fast! Faster!
• Reliable!
• Scalable!
Acrassicau 6 by Bruce Martin, CC BY 2.0, https://www.flickr.com/photos/shotbydarko/4692892946/

6. InstrumentationTests
vs uiautomator
• Base for most frameworks
• Understand the differences!

7. Android Activity Tests
• JUnit3 Tests
• Android Project (android application)
• Tight coupling to app under test (project needed)
• Direct access to activities (find element by id)

8. Android Activity Tests
public class MyActivityTest extends
ActivityInstrumentationTestCase2<MyActivity> {
...
public void testButtons() {
MyActivity activity = getActivity();
Button view = (Button)
activity.findViewById(R.id.button1);
assertEquals("My Button", view.getText());
}

9. Robotium
• Based on Intrumentation Tests
• Removes complexity
• Very popular (used to be the standard)

10. Robotium
public class SimpleActivityTest extends
ActivityInstrumentationTestCase2<MyActivity> {
...
public void setUp() throws Exception {
solo = new Solo(getInstrumentation(), getActivity());
}
public void testPressButton() throws Exception {
solo.clickOnButton("press me");
assertTrue(solo.waitForText("New Window"));

11. UIAutomator
• JUnit3 Tests
• Java Project
• Simple API
• Can access everything on device a user can
• Loose coupling to app under test

12. UIAutomator
public class LaunchSettings extends UiAutomatorTestCase {
public void testPressButton()
throws UiObjectNotFoundException {
getUiDevice().pressHome();
... find and start app
UiObject button = new UiObject(
new UiSelector().text(“clickMe");
button.clickAndWaitForNewWindow();

13. UIAutomator
• Blackbox Testing
• Can only access what is visible
• API 16+

14. InstrumentationTests
vs uiautomator
• Understand the differences
Device
Application
Under Test
uiatuomator
Device
Instrumentaion
Application
Under Test
apps
apps
apps

15. Other Basics...
• Calabash: BDD
Given I am a valid user
And I press "Login"
Then I see "Welcome to coolest app ever”
• Robolectric: unit testing
• Monkey Runner

16. That‘s
not
rock star
testing!
Guitar Hero!!! By JoshBerglund19, CC BY 2.0

17. What about Selenium?
• Any language
• Any test framework
• Selenium Client API (Json Wire Protocol)
• Scale and parallize via Selenium Grid
In Theory:

18. • Used by eBay, Facebook
source: https://github.com/selendroid/selendroid/blob/master/AUTHORS
• Open Source

20. Appium
• Used by
Expedia, LinkedIn, Brands4friends
• Rising Star
• Open Source
• Cross platform
• Based on ui automator

21. Rock
without
Selenium?

22. • Used by: Groupon
• Open Sourced recently
• Remote Control your tests
RoboRemote

23. RoboRemote Architecture
Page 23
Computer/Build Server Device/Emulator
Application
Under Test
RoboRemoteServer
Robotium
UIAutomatorClient
Legend
Groupon Test Engineering Application Developer/Tester Open Source
RoboRemoteClient
UIAutomatorServer
UIAutomator
Tests

24. RoboRemote Sample
• RoboRemote: UIAutomator
UiObject myLabel = new UiObject(
new UiSelector()
.call(“className”, ”android.widget.TextView”)
.call(“text”, “My Text”));
myLabel.call(“click”);
UiObject myLabel = new UiObject(
new UiSelector()
.className(”android.widget.TextView”)
.text(“My Text”));
myLabel.click();
• UIAutomator

25. • Man in the Middle HTTP Proxy
with REST API
• Single war-file
• Open Sourcing planned
RoboRemote + Odo

26. Response Override
• Custom (stub) response
• Modify response contents
• Change HTTP response code
• Add Delay
Odo-ed
(…“soldOut”:“false”,…)(“soldOut”:“true”,…)

27. RoboRemote
• RoboRemote Github:
https://github.com/groupon/robo-remote
• Maven:
http://search.maven.org/ - search|ga|1|
roboremote

28. This was QA view.
What about
developers?

29. • Used by:
Google, XING, mobile.de,
eBay Kleinanzeigen
• Open Sourced October 2013
• Based on Instrumentation
Simple! Reliable! Fast!

30. ViewMatcher
->ViewAction
->ViewAssert
Simple

31. onView(withId(R.id.myview))
.perform(click())
.check(matches(isDisplayed()));
Simple

32. Use Power of Hamcrest Matcher
Simple

33. Waiting is the root of all evil!
Evil face in Ninnaji Temple, Kyoto by MrHicks46, CC BY 2.0, http://www.flickr.com/photos/teosaurio/10138704345/

34. • No „wait()“ in Espresso API!
Speed

35. • Instead:
– Operation on UIThread
– Wait until idle
– Action on UI Thread
– Wait until idle
– Check Result
Speed

36. • What is idle?
– UI Thread
– AsyncTask
– ???? via IdleResource
Speed

37. • Removed flakyness of sleeping
• Improved Testrunner
Reliable

39. • Used by: Google, Paypal, facebook,
Cisco, LinkedIn, eBay, BMW, Intel,
Evernote (source http://testdroid.com/products/success-stories)
• Real devices in the cloud
• Works with:
Espresso, uiautomator, Calabash,
Appium, Robotium + x

40. • Example via Gradle:
...
apply plugin: 'testdroid‘
testdroid {
...
deviceGroup "10_inch_tablets"
deviceLanguageCode "de-DE"
fullRunConfig {
instrumentationRunner =
“... GoogleInstrumentationTestRunner"
}

42. http://testdroid.com/testautomatisierung
Try it and get a free shirt:

43. So what is the
ultimate
rock star test
tool now?

44. Fazit?
Check your needs and your Resources!

45. Let’s rock test!

46. mobile rockstars
• Michael Burton
Creator of Roboguice
Author of „Android Application Development for Dummies“
• Carlos Sessa
Author of „50 Android Hacks“
https://engineering.groupon.com/
https://github.com/groupon

47. More Links
• http://developer.android.com/tools/testing/testing_ui.html
• https://code.google.com/p/robotium/
• https://github.com/calabash/calabash-android
• http://selendroid.io/
• https://github.com/appium/appium
• https://plus.google.com/+StephanLinzner/posts/HhXutBz7tcV-
• https://code.google.com/p/android-test-kit/wiki/Espresso
• https://code.google.com/p/hamcrest/
• https://cloud.testdroid.com