Daniel Knott presents on the challenges of mobile test automation. He discusses different platforms like Android and iOS that have different user interfaces, hardware, and software versions that must be tested. He then outlines solutions like using Robotium for Android test automation, KIF for iOS testing, and crowd sourcing testing by doing internal beta releases.

1. Daniel Knott
XING AG
Challenges in Mobile Test Automation
Agile Testing Days 2011

2. Daniel Knott
Manager Quality Assurance
daniel.knott@xing.com
Daniel Knott has a technical background with different programming languages and quality
assurance tools. After his vocational education at IBM Deutschland GmbH, he studied
Computer Science with a focus on quality assurance. Since 2010 Daniel is working as a
Quality Assurance Manager at XING AG. In different projects he was responsible for the test
management, test automation and test execution in agile teams at XING. Currently, he works
in the mobile team, where he is involved in the test management and test automation on
Android and iPhone devices. Daniel likes to work in agile software development teams and to
automate test cases using technologies such as Robotium, KIF (Keep It Functional),
Selenium and Java. His XING profile: https://www.xing.com/profile/Daniel_Knott

3. 1. Challenges
2. Solutions
3. Conclusion
What’s inside?
2Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011

4. 3Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
Challenges

5. Challenges – Platforms
4Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011

6. Challenges – Platforms
5Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
More
than
one
Vendor
Different
User
Interfaces
Different
Hardware
Different
So7ware
Versions
One
Vendor
Different
Hardware
Different
So7ware
Versions

7. Challenges – I18N
6Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011

8. Challenges – Carrier Network
7Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011

9. 8Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
Solution –
Test Automation

10. Solution – Robotium
9Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
Black
Box
Test
Tool
Based
on
Java
and
JUnit
3
Support
from
Android
1.6
Simulate
User
InteracHon
Simple
Structure
Device
&
Simulator

11. Solution – Robotium
10Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
public
void
testLoginFailed()
throws
Exception
{
solo.enterText(0,
“wrongusername”);
solo.enterText(1,
“wrongpassword”);
solo.clickOnButton(solo.getString(com.xing.android.R.string.button_login));
solo.waitForActivity("com.xing.android.activities.SpinnerLoginActivity",
2000);
assertTrue("ERROR:
Error
message
is
not
shown",
verifyTextIsPresent(solo.getString(
com.xing.android.R.string.dialog_error_body)));
solo.clickOnButton(solo.getString(com.xing.android.R.string.dialog_button));
}

12. Solution – Robotium
11Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
public
void
testLoginFailed()
throws
Exception
{
solo.enterText(0,
“wrongusername”);
solo.enterText(1,
“wrongpassword”);
solo.clickOnButton(solo.getString(com.xing.android.R.string.button_login));
solo.waitForActivity("com.xing.android.activities.SpinnerLoginActivity",
2000);
assertTrue("ERROR:
Error
message
is
not
shown",
verifyTextIsPresent(solo.getString(
com.xing.android.R.string.dialog_error_body)));
solo.clickOnButton(solo.getString(com.xing.android.R.string.dialog_button));
}

13. Solution – Robotium
12Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
public
void
testLoginFailed()
throws
Exception
{
solo.enterText(0,
“wrongusername”);
solo.enterText(1,
“wrongpassword”);
solo.clickOnButton(solo.getString(com.xing.android.R.string.button_login));
solo.waitForActivity("com.xing.android.activities.SpinnerLoginActivity",
2000);
assertTrue("ERROR:
Error
message
is
not
shown",
verifyTextIsPresent(solo.getString(
com.xing.android.R.string.dialog_error_body)));
solo.clickOnButton(solo.getString(com.xing.android.R.string.dialog_button));
}

14. Solution – Robotium
13Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
public
void
testLoginFailed()
throws
Exception
{
solo.enterText(0,
“wrongusername”);
solo.enterText(1,
“wrongpassword”);
solo.clickOnButton(solo.getString(com.xing.android.R.string.button_login));
solo.waitForActivity("com.xing.android.activities.SpinnerLoginActivity",
2000);
assertTrue("ERROR:
Error
message
is
not
shown",
verifyTextIsPresent(solo.getString(
com.xing.android.R.string.dialog_error_body)));
solo.clickOnButton(solo.getString(com.xing.android.R.string.dialog_button));
}

15. Solution – Robotium
14Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011

16. Solution – KIF (Keep It Functional)
15Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
Grey
Box
Test
Tool
ObjecHve
C
Currently
Simulator
only
Simulate
User
InteracHon
Really
Fast
&
Easy
to
integrate

17. Solution – KIF (Keep It Functional)
16Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
+
(id)scenarioLoginWithWrongCredentials
{
KIFTestScenario
*scenario
=
[KIFTestScenario
scenarioWithDescription:@"Test
the
login
wrong
credentials"];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongusername"
intoViewWithAccessibilityLabel:@"Login
User
Name"]];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongpassword"
intoViewWithAccessibilityLabel:@"Login
Password"]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:@"done"]];
[scenario
addStep:[KIFTestStep
stepToWaitForViewWithAccessibilityLabel:LocalizedString
(@"ID_LOGIN_FAILED_MESSAGE")]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:LocalizedString
(@"ACC_FIELD_USER_NAME_LOGIN")]];

18. Solution – KIF (Keep It Functional)
17Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
+
(id)scenarioLoginWithWrongCredentials
{
KIFTestScenario
*scenario
=
[KIFTestScenario
scenarioWithDescription:@"Test
the
login
wrong
credentials"];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongusername"
intoViewWithAccessibilityLabel:@"Login
User
Name"]];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongpassword"
intoViewWithAccessibilityLabel:@"Login
Password"]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:@"done"]];
[scenario
addStep:[KIFTestStep
stepToWaitForViewWithAccessibilityLabel:LocalizedString
(@"ID_LOGIN_FAILED_MESSAGE")]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:LocalizedString
(@"ACC_FIELD_USER_NAME_LOGIN")]];

19. Solution – KIF (Keep It Functional)
18Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
+
(id)scenarioLoginWithWrongCredentials
{
KIFTestScenario
*scenario
=
[KIFTestScenario
scenarioWithDescription:@"Test
the
login
wrong
credentials"];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongusername"
intoViewWithAccessibilityLabel:@"Login
User
Name"]];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongpassword"
intoViewWithAccessibilityLabel:@"Login
Password"]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:@"done"]];
[scenario
addStep:[KIFTestStep
stepToWaitForViewWithAccessibilityLabel:LocalizedString
(@"ID_LOGIN_FAILED_MESSAGE")]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:LocalizedString
(@"ACC_FIELD_USER_NAME_LOGIN")]];

20. Solution – KIF (Keep It Functional)
19Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
+
(id)scenarioLoginWithWrongCredentials
{
KIFTestScenario
*scenario
=
[KIFTestScenario
scenarioWithDescription:@"Test
the
login
wrong
credentials"];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongusername"
intoViewWithAccessibilityLabel:@"Login
User
Name"]];
[scenario
addStep:[KIFTestStep
stepToEnterText:@"wrongpassword"
intoViewWithAccessibilityLabel:@"Login
Password"]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:@"done"]];
[scenario
addStep:[KIFTestStep
stepToWaitForViewWithAccessibilityLabel:LocalizedString
(@"ID_LOGIN_FAILED_MESSAGE")]];
[scenario
addStep:[KIFTestStep
stepToTapViewWithAccessibilityLabel:LocalizedString
(@"ACC_FIELD_USER_NAME_LOGIN")]];

21. Solution – KIF (Keep It Functional)
20Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011

22. 21Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
Solution –
Crowd Source Testing

23. Solution – Internal Beta Release
22Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011

24. 23Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
Conclusion

25. Conclusion
24Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
Know
your
customers
Write
test
automaHon
Combine
tools
Use
crowd
source
tesHng
Don‘t
forget
the
languages
Don‘t
forget
the
carrier
network
Provide
a
high
quality
app

26. Thank you
for your
kind
attention!
The professional network
www.xing.com
Daniel Knott
Manager Quality Assurance
daniel.knott@xing.com
@dnlkntt
http://www.adventuresinqa.com/
25Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011

27. Links & References
26Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
References
 Robotium [http://code.google.com/p/robotium/]
 KIF [http://corner.squareup.com/2011/07/ios-integration-testing.html]
 WaxSim [https://github.com/square/waxsim]
 Hockeykit [http://hockeykit.net/]
Images and Logos
 Apple Logo [http://www.lte-tablets.de/files/2011/08/apple_logo.jpg]
 Android Logo [http://www.android.com/media/wallpaper/gif/android_logo.gif]
 Windows Phone Logo [
http://upload.wikimedia.org/wikipedia/de/thumb/2/23/Windows_Phone_Logo.svg/500px-
Windows_Phone_Logo.svg.png]
 Blackberry Logo [http://us.blackberry.com/legal/bblogo_gray_003.jpg]
 WebOS Logo [http://upload.wikimedia.org/wikipedia/en/e/e0/WebOS_logo.svg]
 Symbian Logo [
http://upload.wikimedia.org/wikipedia/de/thumb/c/c8/Symbian_OS_logo.svg/686px-
Symbian_OS_logo.svg.png]

28. Links & References
27Challenges in Mobile Test Automation | Daniel Knott | Potsdam, 15.11.2011
Images and Logos
 Agile Testing Days Logo [http://agiletestingdays.com/_img/logo_header.png]
 Twitter Logo [https://si0.twimg.com/a/1318896278/images/logos/twitter_newbird_white.png]
 Businesswoman [
http://www.istockphoto.com/stock-photo-16742115-businesswoman-holding-a-blank-sign-
isolated.php]
 Letters, by davide vizzini limbte [http://www.flickr.com/photos/limbte/4007115435/]
 Little Dish, by Juan de Dios Santander Vela [
http://www.flickr.com/photos/juandesant/209098053]
 App Store Logo [http://upload.wikimedia.org/wikipedia/de/0/05/App-Store-Logo.png]
 Robot [http://www.nassaulibrary.org/eastrock/robot.jpg]
 Robotium Logo [
http://f.hatena.ne.jp/images/fotolife/r/re_shikajiro/20110601/20110601224550.jpg]
 Square Logo [
http://c580513.r13.cf2.rackcdn.com/wp-content/uploads/2011/07/
Square_Logo_Landscape1-500x214.png?9d7bd4]