MOVE FAST AND FIX TESTS
Shauvik Roy Choudhary
@shauvik http://shauvik.com
GOING FROM
MANUAL TO AUTOMATED:

MOVE FAST AND FIX TESTS
Shauvik Roy Choudhary
@shauvik http://shauvik.com
GOING FROM
MANUAL TO AUTOMATED:

MOVE FAST AND FIX TESTS
Shauvik Roy Choudhary
@shauvik http://shauvik.com
GOING FROM
MANUAL TO AUTOMATED:
Checks

ABOUT ME
• PhD Thesis:
Cross-platformTesting & Maintenance of
Web & Mobile Apps
• Industry:
Google,Yahoo!, FullStory, Fujitsu Labs, IBM
Research, HSBC, Goldman Sachs
• Entrepreneurship:
GeorgiaTechTI:GER program,VentureLab,
ATDC,TechSquare Labs
Shauvik Roy Choudhary
PhD, GeorgiaTech
Founder, CheckDroid

OUTLINE
• MOTIVATION: Why Automate ?
• MOVE FAST: How to Automate Quickly?
• FIX TESTS: How to Resolve Automation Issues?
• SCENARIO: Android UITesting
• DISCUSSION

WHY AUTOMATETESTS?
Software
ManualTesting

WHY AUTOMATETESTS?
Software
ManualTesting

WHY AUTOMATETESTS?
Software
ManualTesting

WHY AUTOMATETESTS?
Software
ManualTesting

WHY AUTOMATETESTS?
Software
ManualTesting

PLATFORMS
Web Tester

PLATFORMS
Web Tester

PLATFORMS
Web Tester

PLATFORMS
Web Tester

PLATFORMS
Web Tester

PLATFORMS
Web Tester

PLATFORMS
Web Tester

PLATFORMS
Web Tester

Mobile Tester
Lollipop
Android 5.0
9

HOWTO AUTOMATE?

HOWTO AUTOMATE?
WHAT
Parts of your App

HOWTO AUTOMATE?
HOW
Infrastructure
Tool Support
WHAT
Parts of your App

WHAT:TOTEST
• Functional Tests
Positive and negative scenarios
• Non-Functional Tests
Performance, Security, Compatibility…
• Previous Issues (identified Manually)
Never to occur in Regressions

HOW: INFRASTRUCTURE
• Unit Tests
xUnit for different programming languages
• UI Tests
Selenium/WebDriver for Web Apps
Appium for Mobile (iOS,Android)
Espresso for Android

IDEALTESTING PYRAMID
by Mike Cohn

ICE-CREAM CONE

HOUR GLASS
source: just-about.net

TESTING DIAMOND
Source: toddlittleweb.com

CHRISTMASTREE
source: just-about.net

UITESTS
CLOSESTTO
END-USERTESTING

MOVE FAST
How to automate quickly?

RELIABLE INFRASTRUCTURE

VISUALTEST RECORDING

FIX TESTS
How to fix broken tests?

UITESTING ISSUES

UITESTING ISSUES
• FRAGILITY

UITESTING ISSUES
• FRAGILITY
• SPEED

UITESTING ISSUES
• FRAGILITY
• SPEED
• FLAKINESS

1. FRAGILETESTS

REDUCING FRAGILITY
• Use durable selectors
• IDs over XPath
• Relative XPaths over Absolute XPaths
• Other durable labels (class, content-desc..)

2. SLOWTESTS

MAKINGTESTS FAST
• Don’t use Sleeps
• If absolutely necessary, sleep while polling
• Replace slow resources with stubs

doSomething()
Thread.sleep(1000)
checkSomething()

doSomething()
do{
Thread.sleep(1000)
}while(!loaded)
checkSomething()

FLAKINESS
UIThread
TestThread
Work in
Queue

FLAKINESS
UIThread
TestThread
Work in
Queue
click

FLAKINESS
UIThread
TestThread
Work in
Queue
click
Motion
down
Motion
up

FLAKINESS
UIThread
TestThread
Work in
Queue
click
Motion
down
Motion
up
assert

FLAKINESS
UIThread
TestThread
Work in
Queue
click
Motion
down
Motion
up
assert

FLAKINESS
UIThread
TestThread
Work in
Queue
click
Motion
down
Motion
up
assertsleep

FLAKINESS
UIThread
TestThread
Work in Queue
click
Motion
down
Motion
up
assertsleep

TESTS

3. FLAKYTESTS
Image credits: Google, GTAC talk

REDUCING FLAKINESS
• Have a hermetic test environment
Source: googletesting.blogspot.com

REDUCING FLAKINESS
• Have a hermetic test environment
Source: googletesting.blogspot.com

FIX TESTS

1. FRAGILETESTS
FIX TESTS

1. FRAGILETESTS
2. SLOWTESTS
FIX TESTS

1. FRAGILETESTS
2. SLOWTESTS
3. FLAKYTESTS
Image credits: Google, GTAC talk
FIX TESTS

ANDROID UI TESTING

MOBILETESTING PYRAMID
Source: Daniel Knott,AdventuresInQA
Author: Hand-On Mobile AppTesting

ANDROIDTESTINGTOOLS
Image credits: BitBar

ESPRESSO: HOW IT WORKS
UIThread
TestThread
Work in
Queue
Test Case
Motion
down
Motion
up
Espresso
action assert
Blocked

ESPRESSO METHODS
onView(Matcher)
.perform(ViewAction)
.check(ViewAssertion)

EXAMPLES
onView(withId(R.id.greeting))
.perform(click());
onView(withText("Hello Steve!"))
.check(matches(isDisplayed()));

EXAMPLES
onView(withId(R.id.greeting))
.perform(click());
onView(withText("Hello Steve!"))
.check(matches(isDisplayed()));
Find

EXAMPLES
onView(withId(R.id.greeting))
.perform(click());
onView(withText("Hello Steve!"))
.check(matches(isDisplayed()));
Find
Do Stuff

EXAMPLES
onView(withId(R.id.greeting))
.perform(click());
onView(withText("Hello Steve!"))
.check(matches(isDisplayed()));
Find
Do Stuff
Check

onView(Matcher<View>)
● withId
● withText
● withContentDescription
● isDisplayed
● hasFocus
● hasSibling
● ...
● mySpecialMatcher

perform(ViewAction)
● click
● longClick
● doubleClick
● typeText
● scrollTo
● ...
● myCustomAction

check(ViewAssertion)
● matches(...)
● doesNotExist
● myCustomAssertion

HIERARCHY
withParent(Matcher)
withChild(Matcher)
hasDescendant(Matcher)
isDescendantOfA(Matcher)
hasSibling(Matcher)
isRoot()
UI PROPERTIES
isDisplayed()
isCompletelyDisplayed()
isEnabled()
hasFocus()
isClickable()
isChecked()
isNotChecked()
withEffectiveVisibility(…)
isSelected()
ROOT MATCHERS
isFocusable()
isTouchable()
isDialog()
withDecorView(…)
isPlatformPopup()
COMMON HAMCREST
MATCHERS
allOf(Matchers)
anyOf(Matchers)
is(...)
not(...)
endsWith(String)
startsWith(String)
SEE ALSO
Preference matchers
Cursor matchers
USER PROPERTIES
withId(…)
withText(…)
withTagKey(…)
withTagValue(…)
hasContentDescription(…)
withContentDescription(…)
withHint(…)
withSpinnerText(…)
hasLinks()
hasEllipsizedText()
hasMultilineText()
INPUT
supportsInputMethods(…)
hasImeAction(…)
CLASS
isAssignableFrom(…)
withClassName(…)
Matchers
CLICK/PRESS
click()
doubleClick()
longClick()
pressBack()
pressImeActionButton()
pressKey([int/EspressoKey])
pressMenuKey()
closeSoftKeyboard()
openLink(…)
GESTURES
scrollTo()
swipeLeft()
swipeRight()
swipeUp()
swipeDown()
TEXT
clearText()
typeText(String)
typeTextIntoFocusedView(String)
replaceText(String)
POSITION ASSERTIONS
isLeftOf(Matcher)
isRightOf(Matcher)
isLeftAlignedWith(Matcher)
isRightAlignedWith(Matcher)
isAbove(Matcher)
isBelow(Matcher)
isBottomAlignedWith(Matcher)
isTopAlignedWith(Matcher)
LAYOUT ASSERTIONS
noEllipsizedText(Matcher)
noMultilineButtons()
noOverlaps([Matcher])
View Actions
matches(Matcher)
doesNotExist()
selectedDescendantsMatch(…)
View Assertions
onView(Matcher)
.perform(ViewAction)
.check(ViewAssertion)
CHEAT SHEET
2.0

BARISTA
EspressoTest Generation
http://checkdroid.com/barista

DEMO

Tests

TEST CLOUD

WHY AUTOMATETESTS?
Software
ManualTesting

WHY AUTOMATETESTS?
Software
ManualTesting
MOVE FAST
How to automate quickly?

WHY AUTOMATETESTS?
Software
ManualTesting
MOVE FAST
How to automate quickly?
1. FRAGILETESTS
2. SLOWTESTS
3. FLAKYTESTS
Image credits: Google, GTAC talk
FIX TESTS

WHY AUTOMATETESTS?
Software
ManualTesting
MOVE FAST
How to automate quickly?
1. FRAGILETESTS
2. SLOWTESTS
3. FLAKYTESTS
Image credits: Google, GTAC talk
FIX TESTS
ANDROID UI TESTING

DISCUSSION
Are you currently automating tests?
What are some challenges that you see?

THANKS!
Shauvik Roy Choudhary
@shauvik http://shauvik.com