Talk at Test Automation Day 2018, Berlin, Germany. Its about how to integration Appium tests in Android build pipeline and run them on every merge request in Gitlab CI.

1. Continuous mobile automation
in build pipeline
Automation Day Berlin 2018

2. Hello!
I am Dmitry Lemeshko
Mobile QA Automation @ FrontierCarGroup, Berlin
6+ years in Test Automation
AutomateMeNow dmle
2

3. Frontier Car Group
● 6 markets
● 3 web apps
● mobile apps
● ~20 microservices
3

4. Let's start
Mobile test automation from scratch
4

5. Mobile automation stack
5

6. Feature testing process
6
1
Build APK
From feature branch and
point to Staging
environment
Feature testing
Manual testing of new
functionality
2
Merge to master
When passed QA Automated tests on latest
master branch
3
Regression testing
4

7. 7
1
Build APK
Pointing to Staging
environment & upload to
Google Storage
Run tests
Download & install apk,
run tests, analyse results
and report to developers
2
Gitlab CI
Android
pipeline
Test
automation
pipeline

8. First result
● Android/iOS tests in same repo
● Regression testing on master
● Staging environment
● Local emulators
8

9. “
Work done. Any issues?
9

10. “
Bugs are spotted lately
10

11. “
Hard to maintain tests
11

12. “ No full control over test
environment
12

13. “
Hard to scale
13

14. “ Lack of interests from
developers
14

15. Let’s think
What do we want to achieve?
15

16. Continuous
mobile test
automation
It sounds good, right?
16

17. Feature testing process
17
1
Build APK
From feature branch and
point to isolated environment
Regression testing
Automated tests
2
Feature testing
Manual testing of new
functionality
When passed QA
3
Merge to master
4
Build APK
From master branch and
point to Staging
environment
Extended regression
Automated tests on
latest master branch
1 2

18. 1 pipeline instead of 2
18
1
Build APK
Pointing to isolated
environment & uploaded
to Google Storage
Deploy services
Specified branch for
each service, services are
up-and-running
2
Run tests
Take recent APK and run
tests on random device
from market pool
Upload artifacts to gitlab
CI job
3
Attach artifacts
4

19. Steps to do
● Move tests to Android/iOS project repos
● Configure env deployment from tests
● Manage API endpoints in mobile app
● Scale mobile devices & Appium tools
● Build in single android pipeline
19

20. Split & move:
20
Mobile tools iOS testsAndroid tests
● page objects
● tests & hooks
● wd.js wrapper
● setup, logging
● cloud utils

21. 21
Admin Web tests
Projects in
AUT source
code
Dealer Web tests
Customer Web
tests
Dealer iOS tests
Dealer Android
tests
Web tools
Tools
(API/deploy)
Mobile tools
Frameworks
dependencies

22. 22
Dealer iOS tests
Dealer Android tests
Tools (API/deploy) Mobile tools
Project dependencies

23. Configure deployment from tests
● List services and branches
"services": [
{"name": "serviceA", "branch": "master"},
{"name": "serviceB", "branch": "B-270"}
]
● Specify env prefix
"prefix": "automation10"
23

24. Make custom test builds
That point to API with prefix configured before
24

25. 25
Test devices infrastructure
Source: Facebook
In-house farm Cloud services
Virtualization

26. Integrate cloud service
● Upload new builds from Google Storage
● Manage available devices
● Run tests in a single session*
● Handle cloud session time limit
● Set results and pull artifacts
26

27. 27
Gitlab CI pipeline

28. Update pipeline
● Update existing docker image
● Add new gitlab CI runner
● Add new job for tests
● Configure trigger for job
28

29. “ How to add a new job in
Gitlab CI?
29

30. Update .gitlab-ci.yml:
deployAndUItests:
stage: build
script:
- export branch=$(echo $CI_BUILD_REF_NAME)
- bash /buildAndroid.sh
...
- npm run deploy
- export appiumService=testobject env=automation
- npm run regression
30

31. Update .gitlab-ci.yml:
artifacts:
name: "${CI_JOB_NAME}_${CI_COMMIT_REF_NAME}"
when: always
paths:
- tests/uiTests/screenshots
expire_in: 4 weeks
allow_failure: false
only:
- triggers
31

32. Trigger conditions
● Target branch is master
● No WIP in title
● Marked with `Reviewed` label
32

33. “
How does it look in Gitlab CI?
33

34. 34

35. 35

36. Before
● Regression after merge
● Shared unreliable test env
● Self support for appium &
devices
● Tests maintenance is on QA
● Proxying test results
What do we get?
Now
● Regression before merge
● Isolated configurable test env
● Cloud service
● Tests maintenance is on
QA/Dev
● Results in build pipeline
36

37. Flexibility
Any environment, any version of services
Shared maintenance
Developers are fixing tests :)
Fast feedback
Developers get results faster & directly
37

38. 38
iOS project is
in progress...

39. 39
Thanks!
Any questions?
You can find me at AutomateMeNow dmle