Alexey Buzdin "Maslow's Pyramid of Android Testing"

@AlexeyBuzdin
GDGRiga.lv JUG.lv
RigaDevDay.lvCitadele.lv

UI
Tests
Integration
Tests
Unit Tests

UI
Tests
Integration
Tests
Unit Tests
Effort

Effort Cost
UI
Tests
Integration
Tests
Unit Tests

UI
Tests
Integration
Tests
Unit Tests
Effort Cost

Unit Test
๏ Uses simple JUnit
๏ Runs on JVM, not on a device
๏ Lightning fast (5k test in 10
seconds)
๏ Needs Android SDK Stubs

static protected void markConflicting(ArrayList<ScheduleItem> items) {
for (int i=0; i<items.size(); i++) {
ScheduleItem item = items.get(i);
// Notice that we only care about sessions when checking conflicts.
if (item.type == ScheduleItem.SESSION) for (int j=i+1; j<items.size(); j++) {
ScheduleItem other = items.get(j);
if (item.type == ScheduleItem.SESSION) {
if (intersect(other, item, true)) {
other.flags |= ScheduleItem.FLAG_CONFLICTS_WITH_PREVIOUS;
item.flags |= ScheduleItem.FLAG_CONFLICTS_WITH_NEXT;
} else {
// we assume the list is ordered by starttime
break;
}
}
}
}
} https://github.com/google/iosched

@Override
public void displayData(ﬁnal SessionFeedbackModel model,
ﬁnal SessionFeedbackQueryEnum query) {
switch (query) {
case SESSION:
mTitle.setText(model.getSessionTitle());
if (!TextUtils.isEmpty(model.getSessionSpeakers())) {
mSpeakers.setText(model.getSessionSpeakers());
} else {
mSpeakers.setVisibility(View.GONE);
}
AnalyticsHelper.sendScreenView("Feedback: " + model.getSessionTitle());
break;
}
}
https://github.com/google/iosched

java.lang.RuntimeException: Method setText in android.widget.TextView not mocked.
See http://g.co/androidstudio/not-mocked for details.
at android.widget.TextView.setText(TextView.java)
at lv.buzdin.alexey.MainActivityPresenterTest.test(MainActivityPresenterTest.java:39)
@Test
public void name() throws Exception {
TextView textView = new TextView(null);
textView.setText("hello");
}

android {
….
testOptions {
unitTests.returnDefaultValues = true
}
}

Toast.makeText(null, "", Toast.LENGTH_SHORT).show();

Toast.makeText(null, "", Toast.LENGTH_SHORT).show();
Unmockable*

Testable architecture
๏ Use MV* Patterns
๏ Dependency Injection is a must
๏ Try to use POJOs where possible
๏ Wrap static Android classes to Services
(Log, Toast, etc)
๏ Avoid highly coupling to android classes in *
code

Model - View - Whatever
(Activity/Fragment)
๏ Activity is polluted. (Lifecycle
Logic, LayoutInﬂater, static calls,
etc)
๏ We need a POJO

@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
BaseApplication.inject(this);
setContentView(R.layout.activity_screen);
presenter.initPresenter(this);
presenter.initNavigationDrawer();
presenter.openScheduleScreen();
if (presenter.ﬁrstApplicationStart()) {
presenter.openNavigationDrawer();
}
}

public class MainActivityPresenter {
…
public void initPresenter(ActionBarActivity activity) {
this.activity = activity;
ButterKnife.inject(this, activity);
}
public void initNavigationDrawer() {
activity.setSupportActionBar(toolbar);
drawerToggle = new ActionBarDrawerToggle(activity, drawerLayout, R.string.app_name, R.string.app_name) {
@Override
public void onDrawerOpened(View drawerView) {
super.onDrawerOpened(drawerView);
listView.invalidateViews(); //Refresh counter for bookmarks
}
};
drawerToggle.setDrawerIndicatorEnabled(true);
drawerLayout.setDrawerListener(drawerToggle);
activity.getSupportActionBar().setDisplayHomeAsUpEnabled(true);
activity.getSupportActionBar().setHomeButtonEnabled(true);
listView.setAdapter(navigationAdapter);
}
}

Dagger 2
The fastest Java DI Framework!
https://google.github.io/dagger/

@Singleton
public class MainActivityPresenter {
@Inject SocialNetworkNavigationService socialsService;
@Inject SharedPrefsService preferences;
@Inject NavigationAdapter navigationAdapter;
…
public boolean ﬁrstApplicationStart() {
boolean subsequentStart = preferences.getBool(PreferencesConstants.SUBSEQUENT_START);
if (!subsequentStart) {
preferences.setBool(PreferencesConstants.SUBSEQUENT_START, true);
return true;
}
return false;
}
}

public class SharedPrefsService {
@Inject
public Context context;
private SharedPreferences getPrefs() {
return PreferenceManager.getDefaultSharedPreferences(context);
}
public boolean getBool(String key) {
return getPrefs().getBoolean(key, false);
}
public void setBool(String key, boolean value) {
getPrefs().edit().putBoolean(key, value).commit();
}
}

Removes View dependency for Whatever
Button b = (Button)ﬁndViewById(R.id.button);
b.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
b.setBackgroundColor(Red);
}
});

@InjectView(R.id.button)
Button b;
@OnClick(R.id.button)
public void onClick(View v) {
}
Removes View dependency for Whatever

RxBus rxBus = new RxBus();
@InjectView(R.id.button) Button b;
Observable<Void> clicks = RxView.clicks(b);
public init() {
clicks.subscribe(aVoid -> { rxBus.send(new Click()); });
rxBus.toObserverable()
.ﬁlter(e -> e instanceof Click)
.subscribe(e -> {
});
}
https://github.com/JakeWharton/RxBinding
Testable architecture with Rx

@Test
public void test() throws Exception {
MyFragment fragment = new MyFragment();
fragment.b = new Button(null) {
@Override
public void setBackgroundColor(int color) {
assertEquals(color, Red);
}
};
fragment.clicks = Observable.just(null);
fragment.init();
}

Mockito + PowerMock
Mocking, Spying

Object o = mock(Object.class);
doReturn(true).when(o).equals(any());
Object o = spy(“Hi”);
doReturn(true).when(o).equals(any());
o.hashCode() -> is real

@RunWith(MockitoJUnitRunner.class)
public class MyClassTest {
@InjectMocks MyFragment fragment;
@Mock Button b;
@Test
public void test() throws Exception {
fragment.clicks = Observable.just(null);
fragment.init();
verify(b).setBackgroundColor(Red);
}
}

@RunWith(PowerMockRunner.class)
@PrepareForTest( { Toast.class })
public class PowerMockExample {
@Test
public void testPowerMock() throws Exception {
Toast mock = mock(Toast.class);
mockStatic(Toast.class);
when(Toast.makeText(any(), anyString(), anyInt())).thenReturn(mock);
Toast.makeText(null, "1", Toast.LENGTH_SHORT).show();
}
}

JUnit Extra Features
Helpful for Android Developers

JUnit Lifecycle
public class RunnerTest {
@BeforeClass public static void beforeClass() { out.println("Before Class");}
@Before public void before() { out.println("Before");}
@Test public void test() { out.println("Test"); }
@After public void after() { out.println("After"); }
@AfterClass public static void afterClass() { out.println("After Class"); }
}

JUnit Lifecycle
}
Before Class
Before
Test
After
After Class
Process ﬁnished with exit code 0

Custom Runner:
BlockJUnit4ClassRunner
or Runner

public class CustomRunner extends BlockJUnit4ClassRunner{
public static void runnerBefore() { System.out.println("Runner Before");}
public static void runnerBeforeClass() { System.out.println("Runner Before Class"); }
public static void runnerAfter() { System.out.println("Runner After");}
public static void runnerAfterClass() { System.out.println("Runner After Class"); }
@Override protected Statement withBefores(FrameworkMethod method, Object t, Statement st) {
List<FrameworkMethod> list = getFrameworkMethods("runnerBefore");
return new RunBefores(super.withBefores(method, t, st), list, t);
}
….
private List<FrameworkMethod> getFrameworkMethods(String methodName) {
try {
Method runnerBefore = getClass().getDeclaredMethod(methodName);
return Collections.singletonList(new FrameworkMethod(runnerBefore));
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}

@RunWith(CustomRunner.class)
….
}
Runner Before Class
Before Class
Runner Before
Before
Test
After
Runner After
After Class
Runner After Class

….
}
Runner Before Class
Before Class
Runner Before
Before
Test
After
Runner After
After Class
Runner After Class
AndroidJUnitRunner +
MockitoJUnitRunner +
Parametrized

JUnit Rules
•Rules allow ﬂexible addition of the behaviour
of each test method in a test class
•Base Rules Provided in JUnit:
Temporary Folder Rule; ExternalResource Rule;
ErrorCollector Rule; TestName Rule; Timeout Rule;
RuleChain

public class CustomRule implements TestRule {
private boolean classRule;
public CustomRule(boolean classRule) { this.classRule = classRule; }
@Override
public Statement apply(Statement base, Description description) {
return new Statement() {
@Override
public void evaluate() throws Throwable {
System.out.println(classRule ? "Class Rule Before" : "Rule Before");
try { base.evaluate();
} ﬁnally { System.out.println(classRule ? "Class Rule After" : "Rule After”); }
}
};
}
}

@ClassRule public static CustomRule classRule = new CustomRule(true);
@Rule public CustomRule rule = new CustomRule(false);
}

Class Rule Before
Runner Before Class
Before Class
Rule Before
Runner Before
Before
Test
After
Runner After
Rule After
After Class
Runner After Class
Class Rule After

public static class UseRuleChain {
@Rule
public RuleChain chain= RuleChain
.outerRule(new LoggingRule("outer rule")
.around(new LoggingRule("middle rule")
.around(new LoggingRule("inner rule");
@Test
public void example() {
assertTrue(true);
}
}
starting outer rule
starting middle rule
starting inner rule
finished inner rule
finished middle rule
finished outer rule

Test Speciﬁcation
1. Preparation
2. Testable Action
3. Assertion

Test Speciﬁcation
1. Preparation
2. Testable Action
3. Assertion
https://github.com/hamcrest/JavaHamcrest
assertThat(T object, Matcher<T> matcher)

@Test
public void testValidIPAddress() throws InvalidIPAddressException
{
IPAddress addr = new IPAddress("127.0.0.1");
byte[] octets = addr.getOctets();
assertTrue(octets[0] == 127);
}
Bad Test

Examples
•assertThat(2, is(2))
•assertThat(“s”, is(nullValue()))
•assertThat(“s”, is(new String(“s”)))
•assertThat(“s”, equalTo(new String(“s”)))
•assertThat(“s”, not(equalTo(“d”)))
•assertThat(“s”, instanceOf(String.class))
http://hamcrest.org/JavaHamcrest/javadoc/1.3/

List matcher
•assertThat(ids, hasItem(10))
•assertThat(ids, contains(5, 8))
•assertThat(ids, containsInAnyOrder(5, 8))
•assertThat(ids, everyItem(greaterThan(3)))

AllOf AnyOf
assertThat(name, anyOf(startsWith(“A”), endsWith(“B”)))
assertThat(ids, allOf(
hasSize(5),
hasItem(10),
everyItem(greaterThan(3))
))

Error messages
assertThat("s", is(nullValue()))
assertThat(true, is(false))

Error messages
assertThat(1, is(allOf(not(1), not(2), not(10))))

Custom matchers
private Matcher<Foo> hasNumber(final int i) {
return new TypeSafeMatcher<Foo>() {
@Override public void describeTo(final Description description) {
description.appendText("getNumber should return ").appendValue(i);
}
@Override protected void describeMismatchSafely(final Foo item, final
Description mismatchDescription) {
mismatchDescription.appendText(" was ").appendValue(item.getNumber());
}
@Override protected boolean matchesSafely(final Foo item) {
return i == item.getNumber();
}
};
}

hamcrest-rx
Matcher<TestSubscriber<T>> hasValues(final Matcher<? super List<T>> eventsMatcher)
Matcher<TestSubscriber<T>> hasOnlyValues(final Matcher<? super List<T>> values)
Matcher<TestSubscriber<T>> hasOnlyValue(final Matcher<? super T> valueMatcher)
Matcher<TestSubscriber<T>> hasNoValues()
Matcher<TestSubscriber<T>> hasErrors(final Matcher<? super List<Throwable>> err)
Matcher<TestSubscriber<T>> hasNoErrors()
Matcher<TestSubscriber<T>> hasOnlyErrors(final Matcher<? super List<Throwable>> err)
https://github.com/zalando-incubator/undertaking/blob/master/src/test/java/org/
zalando/undertaking/test/rx/hamcrest/TestSubscriberMatchers.java

https://github.com/hertzsprung/hamcrest-json
assertThat(
"{"age":43, "friend_ids":[16, 52, 23]}",
sameJSONAs("{"friend_ids":[52, 23, 16]}")
.allowingExtraUnexpectedFields()
.allowingAnyArrayOrdering());
hamcrest-json

Square Burst
https://github.com/square/burst
public enum Soda { PEPSI, COKE }
public enum Sets {
HASH_SET() {
@Override public <T> Set<T> create() {
return new HashSet<T>();
}
},
LINKED_HASH_SET() { … },
TREE_SET() { … }
public abstract <T> Set<T> create();
}

@RunWith(BurstJUnit4.class)
public class DrinkSodaTest {
@Burst Soda soda;
@Burst Sets sets;
@Test public void drinkFavoriteSodas(Soda soda) {
// TODO Test drink method with 'soda'...
}
}
Square Burst

@RunWith(BurstJUnit4.class)
public class DrinkSodaTest {
private ﬁnal Set<Soda> favorites;
public DrinkSodaTest(Sets sets) {
favorites = sets.create();
}
@Test public void trackFavorites() { // TODO … }
@Test public void drinkFavoriteSodas(Soda soda) { //TODO … }
}
Square Burst

JUnitParams
https://github.com/Pragmatists/JUnitParams
@RunWith(JUnitParamsRunner.class)
public class PersonTest {
@Test
@Parameters({"17, false”, "22, true" })
public void personIsAdult(int age, boolean valid) throws Exception {
assertThat(new Person(age).isAdult(), is(valid));
}
}

Enclosed
Provides a way to use multiple JUnit Runners in a test class

https://www.indiegogo.com/projects/junit-lambda

http://junit.org/junit5/
JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage
import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;
class FirstJUnit5Tests {
@Test
void myFirstTest() {
assertEquals(2, 1 + 1);
}
}

Grouped Assertions
@Test
void groupedAssertions() {
// In a grouped assertion all assertions are executed, and any
// failures will be reported together.
assertAll("address",
() -> assertEquals("John", address.getFirstName()),
() -> assertEquals("User", address.getLastName())
);
}

Throwable Assertions
@Test
void exceptionTesting() {
Throwable exception = assertThrows(IllegalArgumentException.class, () -> {
throw new IllegalArgumentException("a message");
});
assertEquals("a message", exception.getMessage());
}

No assertThat()
import static org.hamcrest.CoreMatchers.equalTo;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.MatcherAssert.assertThat;
import org.junit.jupiter.api.Test;
class HamcrestAssertionDemo {
@Test
void assertWithHamcrestMatcher() {
assertThat(2 + 1, is(equalTo(3)));
}
}

No Runners or Rules > Extensions
@ExtendWith(MockitoExtension.class)
class MyMockitoTest {
@BeforeEach
void init(@Mock Person person) {
when(person.getName()).thenReturn("Dilbert");
}
@Test
void simpleTestWithInjectedMock(@Mock Person person) {
assertEquals("Dilbert", person.getName());
}
}

Dynamic Tests
class DynamicTestsDemo {
@TestFactory
Collection<DynamicTest> dynamicTestsFromCollection() {
return Arrays.asList(
dynamicTest("1st dynamic test", () -> assertTrue(true)),
dynamicTest("2nd dynamic test", () -> assertEquals(4, 2 * 2))
);
}
}
Collection, Iterable, Iterator, Stream

in Android
• Not coming soon
• Would require a rewrite for all testing libraries
• Would require Android Studio support
• Would simplify the Extension model
https://github.com/junit-team/junit5/issues/204

Integration Tests
๏ Uses Robolectric framework
๏ Runs on JVM with Shadow Android SDK
๏ Has access to Context and all Android
peripheral

Robolectric
@RunWith(RobolectricTestRunner.class)
public class MyActivityTest {
@Test
public void clickingButton_shouldChangeResultsViewText() throws Exception {
MyActivity activity = Robolectric.setupActivity(MyActivity.class);
Button button = (Button) activity.ﬁndViewById(R.id.button);
TextView results = (TextView) activity.ﬁndViewById(R.id.results);
button.performClick();
assertThat(results.getText().toString()).isEqualTo("Robolectric Rocks!");
}
}

ActivityController controller = Robolectric.buildActivity(MyAwesomeActivity.class).create().start();
Activity activity = controller.get();
// assert that something hasn't happened
activityController.resume();
// assert it happened!
Activity activity = Robolectric.buildActivity(MyAwesomeActivity.class)
.create().start().resume().visible().get();
Activity Lifecycle

Robolectric
๏ Life saver when complex Android SDK
calls should be tested
๏ Slow compared to Unit Tests
๏ Not up to date to the latest SDKs
(API 24 not supported yet)

UI Tests
๏ Runs on actual Android Device
๏ Slower the Unit tests
๏ Brittle and dependant on
device health

public void testRecorded() throws Exception {
if (solo.waitForText("Hello!")) {
solo.clickOnView(solo.findViewById("R.id.sign_in"));
solo.enterText((EditText) solo.findViewById("R.id.login_username"),"username");
solo.enterText((EditText) solo.findViewById("R.id.login_password"),"password");
solo.clickOnView(solo.findViewById("R.id.login_login"));
solo.waitForActivity("HomeTabActivity");
} solo.clickOnView(solo.findViewById("R.id.menu_compose_tweet") );
solo.enterText((EditText) solo.findViewById(“R.id.edit"), "Testdroid");
solo.clickOnView(solo.findViewById("R.id.composer_post"));
}
Robotium

Android Espresso
@Test 
public void multiActivityTest() { 
onView(withId(R.id.date)) 
.perform(click()); 
// Loads another activity riiiight here 
onView(allOf(withId(R.id.date_expanded), withText("SomeRandomDate"))) 
.check(matches(isDisplayed())) 
.perform(click()); 
// Yay! No waiting! 
}

Espresso comes with Hamcrest integration
@Test 
public void dateTest() { 
onView(withId(R.id.date)) 
.check(matches(withText("2014-10-15"))); 
}

Robotium vs Espresso
• Espresso faster
• Robotium has bigger SDK coverage
• Espresso has built in wait mechanism that is
optimised for android lifecycle
http://www.stevenmarkford.com/android-espresso-vs-robotium-
benchmarks/

Looking into Cross-platform UI Test
Automation?

https://www.youtube.com/watch?v=iwvueGzwVyk

Cross platform tests
If you have dedicated QA team and product on multiple
platforms - go Calabash or Appium
• More ﬂacky tests
• Less performant speed
• Some test reuse
• Easier for QA

How to run UI test
• Don’t initialize run-time dependencies (event
tracking, analytics, long-init things like payment
solutions)
• Don’t hit up real backend, mock out responses
• Insert appropriate test data before test starts
running

How to mock a Server
- Mock Server through DI
- Mock HTTP Server instance on Device
- Dev instance of Server

DI: Create a custom Test Runner
public class MockTestRunner extends AndroidJUnitRunner { 
@Override 
public Application newApplication(ClassLoader cl, String className, Context ctx) 
throws InstantiationException, IllegalAccessException, ClassNotFoundException { 
return super.newApplication(cl, MockDemoApplication.class.getName(), ctx); 
} 
}

DI: Create a custom Test Application
public class MockDemoApplication extends DemoApplication { 
@Override 
protected DemoComponent createComponent() { 
return DaggerMainActivityTest_TestComponent.builder().build(); 
} 
}

MockServer on Device: AndroidAsync
https://github.com/koush/AndroidAsync
AsyncHttpServer server = new AsyncHttpServer();
server.get("/", new HttpServerRequestCallback() {
@Override
public void onRequest(AsyncHttpServerRequest request,
AsyncHttpServerResponse response) {
response.send("Hello!!!");
}
});
server.listen(5000);

Android Jenkins Plugin
https://wiki.jenkins-ci.org/display/JENKINS/Android+Emulator+Plugin

Multi-conﬁguration (matrix) job
Android Jenkins Plugin

https://github.com/Genymobile/genymotion-gradle-plugin
1. genymotion {
2. devices {
3. nexus5 {
4. template "Google Nexus 5 - 4.4.4 - API 19 - 1080x1920"
5. }
6. }
7. }

https://medium.com/@Genymotion/android-os-now-available-as-an-amazon-
machine-image-72748130436b#.njabkxnih

https://github.com/square/spoon
https://github.com/stanfy/spoon-gradle-plugin

Conclusion
๏ Unit tests are cheap, make them your ﬁrst
frontier
๏ Adapt code to make it more testable
๏ Structure tests with @Rules and Hamcrest
Matchers
๏ Mockito + Powermock will help to mock
Android else Robolectric will

Conclusion
๏ UI tests are harder to write and maintain
๏ If you have a dedicated mobile QA team
think of cross-platform tests
๏ For UI tests have a conﬁg for Mocked server
and other integration points
๏ Conﬁgure either emulator startup or device
farm on your CI

TESTS ARE MADE
TO MAKEYOU FEEL SECURE!

Q&A
Thank You!
@AlexeyBuzdinFollow me at

Alexey Buzdin "Maslow's Pyramid of Android Testing"

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to Alexey Buzdin "Maslow's Pyramid of Android Testing"

Similar to Alexey Buzdin "Maslow's Pyramid of Android Testing" (20)

More from IT Event

More from IT Event (20)

Recently uploaded

Recently uploaded (20)

Alexey Buzdin "Maslow's Pyramid of Android Testing"