1. Daniel Knott
XING AG
@dnlkntt
http://www.adventuresinqa.com
Mobile App Testing –
Challenges, Solutions and Best Practices
Scandinavian Agile Conference 2012

2. Daniel Knott
Manager Quality Assurance
@dnlkntt
daniel.knott@xing.com
Daniel Knott has a technical background with different programming languages and quality
assurance tools. After his vocational education at IBM Deutschland GmbH, he studied
Computer Science with a focus on quality assurance. Since 2010 Daniel is working as a
Quality Assurance Manager at XING AG. In different projects he was responsible for the test
management, test automation and test execution in agile teams at XING. Currently, he works
in the mobile team, where he is involved in the test management and test automation on
Android and iPhone devices. Daniel likes to work in agile software development teams and to
automate test cases using technologies such as Robotium, KIF (Keep It Functional),
Selenium and Java. XING profile: https://www.xing.com/profile/Daniel_Knott.
Blog: http://www.adventuresinqa.com

3. 1. XING AG
2. Why mobile testing?
3. Challenges
4. Solutions
5. Best Practices
What’s inside?
2Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012

4. 3Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
XING AG
 www.xing.com
 Business Social Network
 Located in Hamburg
 > 11.7 Mio. Members
 ~20% Mobile Traffic

5. XING AG
4Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012

6. XING AG
5Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
Releases
50

7. XING AG
6Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012

8. 7Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
Why mobile testing?

9. Why mobile testing?
8Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012

10. Why mobile testing?
9Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
App
installa*on
is
failing
Layout
problems
Problems
with
Landscape
/
Portrait
Mode
Wrong
transla*on
Missing
ressources
No
connec*on
to
backend

11. 10Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
Challenges

12. Challenges – Platforms
11Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012

13. Challenges – Platforms
12Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
More
than
one
Vendor
Different
User
Interfaces
Different
Hardware
Different
SoFware
Versions
One
Vendor
Different
Hardware
Different
SoFware
Versions

14. Challenges – I18N
13Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012

15. Challenges – Carrier Network
14Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012

16. 15Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
Solution –
Test Automation

17. Solution – Robotium
16Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
Black
Box
Test
Tool
Based
on
Java
and
JUnit
3
Supports
Android
1.6
–
4.x
Simulate
User
Interac*on
Simple
Structure
Device
&
Simulator
Test
against
APK
files

18. Solution – Robotium
17Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
It‘s likeSelenium,
but for
AndroidTM
clickOnButton(text)
clickOnText(text)
clickOnImage(id)
clickInList(line)
searchText(text)
enterText(id, text)
waitForActivity(name, timeout)
waitForView(view)

19. Solution – Robotium
18Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
public
void
testLoginFailed()
throws
Exception
{
solo.enterText(0,
“wrongusername”);
solo.enterText(1,
“wrongpassword”);
solo.clickOnButton(solo.getString(com.xing.android.R.string.button_login));
solo.waitForActivity("com.xing.android.activities.SpinnerLoginActivity",
2000);
assertTrue("ERROR:
Error
message
is
not
shown",
verifyTextIsPresent(solo.getString(
com.xing.android.R.string.dialog_error_body)));
solo.clickOnButton(solo.getString(com.xing.android.R.string.dialog_button));
}

20. Solution – Robotium
19Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
public
void
testLoginFailed()
throws
Exception
{
solo.enterText(0,
“wrongusername”);
solo.enterText(1,
“wrongpassword”);
solo.clickOnButton(solo.getString(com.xing.android.R.string.button_login));
solo.waitForActivity("com.xing.android.activities.SpinnerLoginActivity",
2000);
assertTrue("ERROR:
Error
message
is
not
shown",
verifyTextIsPresent(solo.getString(
com.xing.android.R.string.dialog_error_body)));
solo.clickOnButton(solo.getString(com.xing.android.R.string.dialog_button));
}

21. Solution – Robotium
20Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
public
void
testLoginFailed()
throws
Exception
{
solo.enterText(0,
“wrongusername”);
solo.enterText(1,
“wrongpassword”);
solo.clickOnButton(solo.getString(com.xing.android.R.string.button_login));
solo.waitForActivity("com.xing.android.activities.SpinnerLoginActivity",
2000);
assertTrue("ERROR:
Error
message
is
not
shown",
verifyTextIsPresent(solo.getString(
com.xing.android.R.string.dialog_error_body)));
solo.clickOnButton(solo.getString(com.xing.android.R.string.dialog_button));
}

22. Solution – Robotium
21Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
public
void
testLoginFailed()
throws
Exception
{
solo.enterText(0,
“wrongusername”);
solo.enterText(1,
“wrongpassword”);
solo.clickOnButton(solo.getString(com.xing.android.R.string.button_login));
solo.waitForActivity("com.xing.android.activities.SpinnerLoginActivity",
2000);
assertTrue("ERROR:
Error
message
is
not
shown",
verifyTextIsPresent(solo.getString(
com.xing.android.R.string.dialog_error_body)));
solo.clickOnButton(solo.getString(com.xing.android.R.string.dialog_button));
}

23. Solution – Robotium
22Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012

24. Solution – KIF (Keep It Functional)
23Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
Grey
Box
Test
Tool
Objec*ve
C
Currently
Simulator
only
Simulate
User
Interac*on
Really
Fast
&
Easy
to
integrate

25. Solution – KIF (Keep It Functional)
24Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
stepToEnterText: @”Text”
stepToTapViewWithAccessibilityLabel: @”Text”
stepToWaitForViewWithAccessibilityLabel: @”Text”
stepToWaitForTimeInterval: 1
stepToWaitForTappableViewWithAccessibilityLabel: @”Text”
stepToWaitForAbsenceOfViewWithAccessibilityLabel: @”Text”
stepToTapRowInTableViewWithAccessibilityLabel: @”Text”

26. Solution – KIF (Keep It Functional)
25Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
+
(id)scenarioLoginWithWrongCredentials
{
KIFTestScenario
*scenario
=
[KIFTestScenario
scenarioWithDescription:@"Test
the
login
wrong
credentials"];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongusername"
intoViewWithAccessibilityLabel:@"Login
User
Name"]];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongpassword"
intoViewWithAccessibilityLabel:@"Login
Password"]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:@"done"]];
[scenario
addStep:[KIFTestStep
stepToWaitForViewWithAccessibilityLabel:LocalizedString
(@"ID_LOGIN_FAILED_MESSAGE")]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:LocalizedString
(@"ACC_FIELD_USER_NAME_LOGIN")]];

27. Solution – KIF (Keep It Functional)
26Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
+
(id)scenarioLoginWithWrongCredentials
{
KIFTestScenario
*scenario
=
[KIFTestScenario
scenarioWithDescription:@"Test
the
login
wrong
credentials"];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongusername"
intoViewWithAccessibilityLabel:@"Login
User
Name"]];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongpassword"
intoViewWithAccessibilityLabel:@"Login
Password"]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:@"done"]];
[scenario
addStep:[KIFTestStep
stepToWaitForViewWithAccessibilityLabel:LocalizedString
(@"ID_LOGIN_FAILED_MESSAGE")]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:LocalizedString
(@"ACC_FIELD_USER_NAME_LOGIN")]];

28. Solution – KIF (Keep It Functional)
27Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
+
(id)scenarioLoginWithWrongCredentials
{
KIFTestScenario
*scenario
=
[KIFTestScenario
scenarioWithDescription:@"Test
the
login
wrong
credentials"];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongusername"
intoViewWithAccessibilityLabel:@"Login
User
Name"]];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongpassword"
intoViewWithAccessibilityLabel:@"Login
Password"]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:@"done"]];
[scenario
addStep:[KIFTestStep
stepToWaitForViewWithAccessibilityLabel:LocalizedString
(@"ID_LOGIN_FAILED_MESSAGE")]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:LocalizedString
(@"ACC_FIELD_USER_NAME_LOGIN")]];

29. Solution – KIF (Keep It Functional)
28Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
+
(id)scenarioLoginWithWrongCredentials
{
KIFTestScenario
*scenario
=
[KIFTestScenario
scenarioWithDescription:@"Test
the
login
wrong
credentials"];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongusername"
intoViewWithAccessibilityLabel:@"Login
User
Name"]];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongpassword"
intoViewWithAccessibilityLabel:@"Login
Password"]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:@"done"]];
[scenario
addStep:[KIFTestStep
stepToWaitForViewWithAccessibilityLabel:LocalizedString
(@"ID_LOGIN_FAILED_MESSAGE")]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:LocalizedString
(@"ACC_FIELD_USER_NAME_LOGIN")]];

30. Solution – KIF (Keep It Functional)
29Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012

31. 30Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
Solution –
Crowd Source Testing

32. Solution – Internal Beta Release
31Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012

33. Solution – Crowd Source Testing
32Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012

34. Solution – Cloud Testing
33Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012

35. 34Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
Best
Practices

36. Best Practices
35Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
Know
your
customers
Write
test
automa*on
Combine
tools
Use
crowd
source
tes*ng
Don‘t
forget
the
languages
Don‘t
forget
the
carrier
network
Test
code
and
produc*on
code
in
same
language
Use
version
control
systems
Use
con*nuous
integra*on
Avoid
simulators

37. Thank you
for your
kind
attention!
The professional network
www.xing.com
Daniel Knott
Manager Quality Assurance
daniel.knott@xing.com
@dnlkntt
http://www.adventuresinqa.com/
36Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012

38. We are hiring
37Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
We are looking for
Ruby, Perl, Java, JavaScript, iOS, QA
http://corporate.xing.com/english/company/careers-at-xing/

39. Links & References
38Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
 Robotium [http://code.google.com/p/robotium/]
 KIF [http://corner.squareup.com/2011/07/ios-integration-testing.html]
 WaxSim [https://github.com/square/waxsim]
 Hockeykit [http://hockeykit.net/]
 Apkudo [http://www.apkudo.com/]
 uTest [http://www.utest.com/]
 Testbirds.de [http://testbirds.de/]
 Testdroid [http://testdroid.com/]
 Perfecto Mobile [http://www.perfectomobile.com/]
 DeviceAnyWhere[http://www.keynotedeviceanywhere.com/index.aspx]

40. Links & References
39Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
 Scandinavian Agile Logo [http://scan-agile.org/wp-content/themes/sac20121/images/
scan_agile_2012_gray.png]
 Twitter Logo [https://si0.twimg.com/a/1318896278/images/logos/twitter_newbird_white.png]
 Businesswoman [http://www.istockphoto.com/stock-photo-16742115-businesswoman-
holding-a-blank-sign-isolated.php]
 Letters, by davide vizzini limbte [http://www.flickr.com/photos/limbte/4007115435/]
 Little Dish, by Juan de Dios Santander Vela [http://www.flickr.com/photos/juandesant/
209098053]
 App Store Logo [http://upload.wikimedia.org/wikipedia/de/0/05/App-Store-Logo.png]
 Robot [http://www.nassaulibrary.org/eastrock/robot.jpg]
 Robotium Logo [http://f.hatena.ne.jp/images/fotolife/r/re_shikajiro/20110601/
20110601224550.jpg]
 Square Logo [http://c580513.r13.cf2.rackcdn.com/wp-content/uploads/2011/07/Square_Logo
_Landscape1-500x214.png?9d7bd4]
 uTest Logo [http://upload.wikimedia.org/wikipedia/en/0/0d/UTest_company_logo.jpg]
 Mob4Hire [http://www.free-press-release.com/members/members_pic/200901/img/
1233074965.png]
 Apkudo [http://www.apkudo.com/a/images/apkudo.png]
 Testbirds [http://nest.testbirds.de/sites/default/files/LOGO_1.png]

41. Links & References
40Mobile App Testing – Challenges, Solutions and Best Practices | Daniel Knott @dnlkntt | Helsinki, 08.03.2012
 DeviceAnywhere [http://www.keynotedeviceanywhere.com/images/logo.jpg]
 Perfecto Mobile Logo [http://www.perfectomobile.com/portal/export/sites/qsp/_info_/
general_image_gallery/logo.gif]
 Testdroid Logo [http://beta.testdroid.com/images/logo.png]
 Apple Logo [http://www.lte-tablets.de/files/2011/08/apple_logo.jpg]
 Android Logo [http://www.android.com/media/wallpaper/gif/android_logo.gif]
 Windows Phone Logo [http://upload.wikimedia.org/wikipedia/de/thumb/2/23/Windows_
Phone_Logo.svg/500px-Windows_Phone_Logo.svg.png]
 Blackberry Logo [http://us.blackberry.com/legal/bblogo_gray_003.jpg]
 WebOS Logo [http://upload.wikimedia.org/wikipedia/en/e/e0/WebOS_logo.svg]
 Symbian Logo [http://upload.wikimedia.org/wikipedia/de/thumb/c/c8/Symbian_OS_
logo.svg/686px-Symbian_OS_logo.svg.png]
 Google Play Logo [
http://www.android.com/images/brand/android_app_on_play_logo_small.png]
 Apple App Store Logo [http://www.inext.me/wp-content/uploads/2011/09/
app-store_logo.png]
 Windows Market Place Logo [http://modmyi.com/attachments/forums/iphone-4-
new-skins-themes-launches/483774d1290368882-release-windows-phone-7-hd-market-
place-2x.png]