1.
CLOSE TO THE METAL
WHY AND HOW WE USE XCTEST & ESPRESSO
Mario Negro Ponzi – CoE Software Development and Tooling

2.
THE APPS
Mobile Banking Grip Tikkie &Meer

3.
PlanPrinciples
SUMMARY
Execution

4.
PlanPrinciples
SUMMARY
Execution

5.
GITFLOW PROBLEMSTHE PRINCIPLES
1. Anytime releases
2. DevOps teams
3. Infrastructure

6.
Release
as often
as possible
ANYTIME RELEASES

7.
Teams
must be
fully
autonomous
DEVOPS TEAMS

8.
GITFLOW PROBLEMSINFRASTRUCTURE
Building
infrastructure
is not
our core
competence

9.
GITFLOW PROBLEMSINFRASTRUCTURE

10.
PlanPrinciples
SUMMARY
Execution

11.
GITFLOW PROBLEMSTHE PLAN
1. What we tried
2. Tooling
3. CI/CD

12.
• We did build a dedicated
test automation team
• Get the business people
write the tests
• Aim at cross-platform
compatibility
WHAT WE TRIED

13.
WHAT WE TRIED – GHERKIN
Feature: Google Searching
As a web surfer, I want to search Google, so that I can
learn new things.
Scenario: Simple Google search
Given a web browser is on the Google page
When the search phrase "ABN AMRO" is entered
Then results for "ABN AMRO" are shown

14.
15
TOOLING

15.
16
TOOLING
What do we
lose?

16.
17
CI/CD
Automation also means
unessential custom
steps (tools) must be
avoided to reduce the
complexity of the
pipeline

17.
PlanPrinciples
SUMMARY
Execution

18.
1. A third party must manage the devices
2. User management per team and individual
developers and testers allowing for
DevOps
3. Support for XCtest and Espresso
WE USE SAUCE LABS BECAUSE…

19.
Web testing…
😀
BONUS POINTS

20.
• UI Tests
• E2E tests
KIND OF TESTS

21.
THE PIPELINES
Main
runs unit and smoke tests
(on every commit)
UI
runs UI tests (hourly)
E2E
runs E2E tests (nightly)

22.
It does everything you would expect from a ‘normal’ pipeline
• It builds the apps
• It runs unit tests
• It performs a number of quality and security checks
• It runs smoke tests*
• It notifies via Slack
MAIN PIPELINE
* Some parts are still not completed

23.
It checks that the UI looks right on multiple devices
• It runs hourly
• It runs UI tests on multiple random devices (4 + reference device)
• It notifies via Slack
• It does not require a server
• It will not require a Mac to run (even for iOS)
UI TESTS PIPELINE *
* Currently implemented only on Android

24.
UI TESTS EXAMPLE

25.
The real deal
• It runs nightly
• It runs E2E tests against a mock server
• The mock server runs on AWS Lambda and emulates Wiremock
• It notifies via Slack
• It will not require a Mac to run (even for iOS)
• If successful it will re-run the tests against our test environment
E2E TESTS PIPELINE *
* Not fully working, yet

26.
E2E TESTS EXAMPLE

27.
java -jar runner.jar android
--test <test.apk>
--app <app.apk>
--apikey <my project API key>
--datacenter EU
--device Samsung_Galaxy_Note8_ab
A SIMPLE COMMAND

28.
Photos by various
authors on Unsplash