This document discusses various types of tests for Android applications, including instrumentation tests, which run on an emulator or physical device, and unit tests, which run on a JVM without Android dependencies. It covers challenges with instrumentation tests like speed and dependencies on the device state. The document recommends writing business logic separately from UI code to make it more testable. It also provides information on frameworks like Robolectric, Mockito, and JaCoCo that can help with unit testing and code coverage of Android applications.

1. Unit Testing & Android
Tomáš Kypta

2. Unit Testing
Experience on Android

4. Android apps are
difficult to test

6. Types of Android tests

7. Types of Android tests
Instrumentation
tests
Unit tests

8. Instrumentation tests

9. Instrumentation tests
• running on physical device or emulator
• gradle connectedCheck

11. Intstrumentation tests
• Legacy instrumentation tests
or
• Testing Support Library

12. Legacy instrumentation tests
• JUnit3
• Tests extend from TestCase
• AndroidTestCase
• ActivityInstrumentationTestCase2
• ServiceTestCase
• …

13. Testing Support Library
• AndroidJUnitRunner
• JUnit4 compatible
android {
defaultConfig {
testInstrumentationRunner
"android.support.test.runner.AndroidJUnitRunner"
}
}
androidTestCompile 'com.android.support.test:runner:0.3'
androidTestCompile 'com.android.support.test:rules:0.3'

14. Testing Support Library
• Test filtering
• @RequiresDevice - run on physical device, no
emulator
• @SdkSupress - don’t run on lower Android API
level

15. Test Sizes
• @SmallTest
• @MediumTest
• @LargeTest

16. Test Sizes
Feature Small Medium Large
Network access No localhost only Yes
Database No Yes Yes
File system access No Yes Yes
Use external systems No Discouraged Yes
Multiple threads No Yes Yes
Sleep statements No Yes Yes
System properties No Yes Yes
Time limit (seconds) 60 300 900+

17. Potential problems
• Manifest merger
• problems when testing libraries
• aar dependency using manifest placeholders
• e.g. ${applicationId}, ${localApplicationId}
android {
defaultConfig {
manifestPlaceholders =
[localApplicationId:”com.example.mylib”]
}
}

18. Problems
• It affects the device!
• Wiping contacts will wipe contacts!
• You can’t prepare all test preconditions
• e.g. you can’t dynamically change permissions

19. Problems
Test failed to run to completion. Reason: 'Instrumentation
run failed due to 'java.lang.IllegalStateException''.
Check device logcat for details

21. • framework for functional UI tests
• part of Android Testing Support Library
androidTestCompile
'com.android.support.test.espresso:espresso-core:2.2'

22. @Test
public void sayHello(){
onView(withId(R.id.editText))
.perform(typeText(STRING_TO_BE_TYPED),
closeSoftKeyboard());
onView(withText("Say hello!”))
.perform(click());
String expectedText =
"Hello, " + STRING_TO_BE_TYPED + "!";
onView(withId(R.id.textView))
.check(matches(withText(expectedText)));
}

23. Problems
• testing on device is not isolated
• device state affects the result
• e.g. screen on/off might affect test result
onView(withId(R.id.my_view))
.check(matches(isDisplayed()));

24. Instrumentation tests
are
SLOOOOOW

25. java.lang.RuntimeException: Stub!

26. Unit Tests

27. Unit Tests
• run on JVM
• mockable android.jar
• gradle test

28. Grade and Android Studio
support
• natively supported since Gradle plugin 1.1.0
• big problems in AS in previous versions
• has issues
• switching between unit & instrumentation tests
• disabled type is not indexed

31. • the essential piece
• alone can be used only for pure Java
• don’t use on Android APIs!

32. Method ... not mocked.
android {
testOptions {
unitTests.returnDefaultValues = true
}
}
• Helps rarely
• returns 0, false, null, …

34. • mocking framework
• easy to use
• compatible with Android unit testing
testCompile ‘org.mockito:mockito-core:1.10.19'

35. • can be used in instrumentation tests
• needs dexmaker
androidTestCompile ‘org.mockito:mockito-core:1.10.19'
androidTestCompile "com.google.dexmaker:dexmaker:1.2"
androidTestCompile "com.google.dexmaker:dexmaker-mockito:1.2"

36. • @RunWith(MockitoJUnitRunner.class)
• @Mock
• Mockito.mock()
• Mockito.when().thenReturn()
• when(obj.getA()).thenReturn(aInstance)

37. • Mockito.spy()
• wrapping a real object
• Mockito.verify()
• verify that special condition are met
• e.g. method called, called twice, …

38. Limitations
• final classes
• anonymous classes
• primitive types
• static methods

40. • at first, might be difficult to use
• the ultimate mock of Android APIs
• allows custom shadows
• @RunWith(RobolectricTestRunner.class)

41. • supports Android API level 18
• doesn’t support multiple users on the same
machine!!!
2.4

42. • Robolectric class splits into
• Robolectric
• RuntimeEnvironment
• Shadows
• ShadowApplication
• ShadowLooper
3.0-rc3

43. that’s the s**t you need for
Android unit testing

44. Potential problems
• difficult to search for solutions
• long history of bigger changes
• many obsolete posts

45. Potential problems
• difficulties running on command line and in AS
• different paths
• difficulty working with resources
• RobolectricGradleTestRunner
• doesn’t work in AS

46. Code Coverage

47. Code Coverage
• unit tests
• JaCoCo
• instrumentation tests
• EMMA
• obsolete
• Google is supposedly working on JaCoCo support

48. JaCoCo

49. JaCoCo
• enabled by default for unit tests
• generates binary report in build/jacoco
• build/jacoco/testDebugUnitTest.exec
• gradle test

52. Code Coverage
• don’t use it
• generates coverage-instrumented-classes
• for instrumentation tests
• coverage for instrumentation tests is not ready
buildTypes {
debug {
testCoverageEnabled true
}
}

53. Good tests

54. Good tests
• run in any order
• run in isolation
• run consistently
• run fast
• are orthogonal

55. How to write testable
apps?

56. Rules of thumb
• prefer pure Java
• abstract away from Android APIs
• separate business logic and UI
• don’t write business logic into activities and
fragments
• try avoid static methods
• use dependency injection

57. Questions?

58. References
• Espresso
• https://code.google.com/p/android-test-kit/
• https://code.google.com/p/android-test-kit/wiki/
EspressoV2CheatSheet

59. References
• Mockito
• http://mockito.org/
• https://github.com/mockito/mockito
• Dexmaker
• https://github.com/crittercism/dexmaker

60. References
• Robolectric
• http://robolectric.org/
• https://github.com/robolectric/robolectric

61. References
• code coverage
• JaCoCo
• Eclipse Public License v1.0
• http://www.eclemma.org/jacoco/trunk/index.html
• https://github.com/jacoco/jacoco
• EMMA
• Common Public License v1.0
• http://emma.sourceforge.net/