Sathish Gogineni, Badoo, QA-секция CodeFest 2015

1. Badoo:
Cross
pla,orm
mobile
test
automa3on
and
con3nuous
delivery
Sathish
Gogineni
Mobile
QA
Automa4on
Lead

3. Raise
your
hands
if….
• You’re
working
in
Mobile
pla=orm?
• You
have
apps
on
mul4ple
mobile
pla=orms?
• You’re
aware
of
the
test
automa4on
tools
in
Mobile?
• You
are
using
func4onal
test
automa4on?
• You’re
using
“Cross
pla=orm”
test
automa4on?

4. A
bit
of
self
promo3on

5. Who
is
Badoo?
• A
social
networking
service
• 241
million
users
• Available
in
190
countries.
• Support
45
languages
on
mobile
• 250
employees
based
in
London
and
Moscow

6. More
about
Badoo
• Badoo
has
2
popular
products
• Apps
support
4
mobile
pla=orms
– Badoo
– HotOrNot

7. Mobile
Test
Automa3on
at
Badoo
• A
single
test
automa4on
code
base
– Run
across
two
products
– Run
on
three
pla=orms
– Run
on
phones
and
tablets
• We
have
– 68%
func4onal
coverage
for
Badoo
in
Android
&
iOS
(
390
scenarios
)
– 76%
func4onal
coverage
for
HotOrNot
in
Android
&
iOS
(
250
scenarios
)
– 52%
func4onal
coverage
in
MobileWeb
(
165
scenarios
)
– And
in
Windows
pla=orm
we
have
0%

9. A
bit
about
Cross
pla,orm
test

10. Mobile
cross
pla,orm
test
automa3on?
• Write
once
,
Run
anywhere(?)
• Write
test
specifica4ons
• Implement
the
tests
at
high
level
• Implement
the
pla=orm/applica4on
UI
code

11. Cross
pla,orm
stack
in
BDD
Scenarios
Feature
Step
defini3ons
Android
Page
iOS
Page
Test
Specifica4on
High
level
implementa4on
Pla=orm
specific

12. Why
cross
pla,orm
test
automa3on?
• Business
logic
is
same.
It
is
only
interac4on
with
UI
is
change
• Pooled
resources
• More
reusable
code.
Less
maintenance

13. What
should
be
considered?
• Who
is
wri4ng
the
test
cases?
App
developers
/
QA
developers?
• Which
mobile
pla=orms?
• Wri4ng
in
language
other
than
pla=orm
specific

15. A
bit
about
Cross
Pla,orm
test
tools

16. Mobile
Cross
Pla,orm
Tools
• Instrument
based
tools
• Image
Recogni4on
tools
– Calabash
– Appium
– Monkey
Talk
– eggPlant
– seetest

17. A
bit
about
Calabash

18. What
is
Calabash?
• A
framework
wriben
in
“Ruby”
• Has
client-­‐server
architecture
using
JSON
• Integrated
with
cucumber
• Supports
Android
and
iOS
pla=orms

19. Calabash
:
Pros
&
Cons
• Pros
– Has
well
structured
client
framework
– Supports
backdoor
opera4ons
to
interact
with
Applica4on
code
• Cons
– Can’t
run
on
“Produc4on”
build
in
iOS
– Have
to
use
Ruby
programming
language
– Forced
to
write
test
cases
in
BDD
(
‘Given
When
Then’
)

20. A
bit
about
Appium

21. What
is
Appium?
• Is
a
Server
wriben
in
Node.js
using
JavaScript
• Uses
pla=orm-­‐provided
frameworks
under
the
hood
• Supports
Web
driver
JSONWire
protocol
• Supports
iOS,
Android,
Mobile
Web
and
Hybrid
applica4ons

22. Appium
:
Pros
&
Cons
• Pros
– Tests
can
be
wriben
in
any
(
most
of
)
programming
languages
– Good
community
support
– Can
use
same
produc4on
builds
for
tes4ng
• Cons
– Lible
bit
“buggy”

24. A
bit
about
Con4nuous
Delivery

25. Con3nuous
Delivery
is…
A
prac4ce
of
crea4ng,
tes3ng
and
releasing
stable
applica4on
frequently
in
smaller
chunks

26. What
is
Con3nuous
Delivery?
Con3nuous
Delivery
=
Con3nuous
Integra3on
+
Con3nuous
Test
Automa3on

27. Con3nuous
Test
Automa3on
Development
Branch

28. Con3nuous
Test
Automa3on
Development
Branch

29. Con3nuous
Test
Automa3on
Development
Branch
Implement
Feature
tests

30. Con3nuous
Test
Automa3on
Development
Branch
Feature
Tests
Implement
Feature
tests

31. Con3nuous
Test
Automa3on
Development
Branch
Feature
Tests
Smoke
Tests
Implement
Feature
tests

32. Con3nuous
Test
Automa3on
Development
Branch
Feature
Tests
Smoke
Tests
Implement
Feature
tests

33. Con3nuous
Test
Automa3on
Development
Branch
Feature
Tests
Smoke
Tests
Nightly
Builds
Nightly
Builds
Implement
Feature
tests

34. Con3nuous
Test
Automa3on
Development
Branch
Feature
2
Feature
1

35. Con3nuous
Test
Automa3on
Development
Branch
Master
Feature
2
Feature
1

36. Con3nuous
Test
Automa3on
Development
Branch
Integra4on
Branch
Master
Feature
2
Feature
1

37. Con3nuous
Test
Automa3on
Development
Branch
Integra4on
Branch
Integra4on
Tests
Master
Feature
1
Feature
2

38. Con3nuous
Test
Automa3on
Development
Branch
Integra4on
2
Integra4on
1
Master
Branch

39. Con3nuous
Test
Automa3on
Development
Branch
Integra4on
2
Integra4on
1
Master
Branch
Nightly
Builds
Nightly
Builds

40. Con3nuous
Test
Automa3on
Development
Branch
Integra4on
2
Integra4on
1
Master
Branch
Release
Branch

41. Con3nuous
Test
Automa3on
Development
Branch
Integra4on
2
Release
Tests
Integra4on
1
Master
Branch
Release
Branch

42. Con3nuous
Test
Automa3on
Development
Branch
Integra4on
2
Integra4on
1
Master
Branch
Release
Branch
Release
Tests
Release
Tests

44. Scaling
build
infrastructure

45. How
we
started?
• A
single
build
machine
with
Jenkins
• Running
on
Android
emulator
and
iOS
simulator
• Run
the
tests
only
on
“master”
branch
every
night

47. What
we
have
today?
• Distributed
CI
server
with
12
mac-­‐mini
agents
• Each
Agent
has
Android
and
iOS
device
connected
• Running
the
tests
on
Real
devices
• Integrated
with
JIRA

50. What
we
will
do
in
future?
• More
devices
to
each
agent
and
Parallelizing
test
suite
• Using
VMs
for
scaling
build
agents

51. Summary
• Worth
wri4ng
cross
pla=orms
tests.
It
Ensure
behavior
is
same
in
every
pla=orm
• Mobile
Test
Automa4on
tools
are
s4ll
evolving
• And,
they
have
limita4ons

52. Nevertheless…
• We
reduced
Regression
tes4ng
to
a
day
for
every
release
(
66%
4me
save
)
• 3X
more
releases
to
the
app
stores
• More
stable
applica4on.
Crash
rate
reduced
from
6%
to
2%

53. hQps://techblog.badoo.com/
Sathish
Gogineni
sathish.gogineni@corp.badoo.com