Talk from Droidcon Zagreb 2015 about approach to Continous Integration and integration testing we are using in Outline. In short: we use Jenkins CI on Intel NUC to build, analyse and test apps. To speed up tests we are using Spoon with some devices connected to our CI server via USB hub.

1. Continous UI testing
by
with Espresso and Jenkins

2. Sylwester Madej
Founder & Head Developer
@smdremedy
#droidconZG
sylwester.madej@byoutline.com
byoutline.com
2

3. This is a story of
my team's journey

4. ...jurney to CI/CD
We are a small team

5. ...jurney to CI/CD
We are a small team
- we are looking for things to be cost effective
- not a perfect solution, but a decent one

6. What we have gained
- better, more agile work flow
- no more Commit and run situations
- way to check UI on many configurations
- less regression bugs
- higher code quality

7. We started with tests
- JVM tests (finally works in AS)
- Robolectric
- Instrumentation

8. We wanted more from VCS
- Use DVCS
- Gitflow = git++
- Branch per feature, tag per release

9. Static code analysis
- FindBugs
- PMD
- Checkstyle
- SonarCube

10. Continous Integration checklist
- gitflow ✔
- unit tests ✔
- code coverage ✔
- static analysis ✔
- build with one command ✔

11. We needed our CI server
- Jenkins
- Travis
- Circle CI
- TeamCity
- Bamboo...

12. Why Jenkins?
- free of change, but not of hustle
- great community
- can be hosted on any device
- hundreds of plugins

13. Where to host it?
- CloudBees.com
- VPS: s3, digital ocean
- bare metal

14. Perfect local Jenkins machine
- good price/performance (i5, ssd, ram)
- energy effective
- quiet
- small
Candidates:
- Intel NUC (7W idle)
- Mac Mini

15. CI setup
Ubuntu
- apt-get install
- sdk install (Android Plugin don't install everything)
- kvm
- plugins (Chuck Norris, Lint Publisher, jacoco...)

16. First easy win
- create new project
- add git with credentials
./gradlew build test jacoco
You get:
- compilation check
- lint check
- unit tests with code coverage

17. We wanted more
Instrumentation tests
- Robotium
- Appium
- Espresso
- UI Automator

18. Some Espresso
Great way to test UI, and do integration tests
Api from developers to developers
onView(Matcher)
.perform(ViewActions)
.check(ViewAssertions)
More on Maciej's speach at 12:00:
„Expressive functional testing with Espresso”

19. How to test API?
Mock/stub API classes
- Dagger FTW
- works great with Retrofit

20. How to test API?
MockWebServer
- set HTTP mock for one test
- Square Quality™, build upon OkHttp
- mock calls, verify where they called

21. How to test API?
AndroidStubServer
- HTTP server build in your APK (good for integration testing )
- can be sent to client within app
simple config:
{
"method": "GET",
"path": "/books",
"response file": "books.json"
}
https://github.com/byoutline/AndroidStubServer

22. Where to run Instrumentation tests?
- androd emulators (x86 if possible)
- genymotion
- real devices

23. Start simple
- create job with single emu
- use Android Emulator Plugin for configuration
- start instrumentation tests in gradle
./gradlew connectedCheck

24. Enter the matrix
- create Multi-configuration project
- exclude combinations

25. Need a Spoon for your Espresso?
- great tool from Square (duh !)
- run instrumentation tests on multiple devices in parallel
- external plugin to run with gradle
https://github.com/stanfy/spoon-gradle-plugin
- connect your devices to CI server with USB hub
- generates nice looking reports

26. One step further - Continous Delivery
- your code is tested and ready to meet the world
- create new job to deploy to distibution channel
Crashlytics
Google Play Alpha/Beta
- start after succesful build or manualy

27. When we need more devices
Devices in the cloud
- AppThwack
- TestDroid...
Bonus:
Some free Intel devices

28. How to make it even better?
Genymotion on server
http://www.slideshare.net/vishyrich/genymotion-with-jenkins

29. BDD with Calabash
Black box testing
calabash-android gen
write specification with Gherkin DSL
Scenario: As a valid user I can log into my app
When take picture
When I press "Login"
Then I see "Welcome to coolest app ever"
Then take picture
calabash-android run APK
Works great with AndroidStubServer!

30. Sylwester Madej
Founder & Head Developer
@smdremedy
sylwester.madej@byoutline.com
byoutline.com
3
0