6. my.name // Kazuaki Matsuo
my.role // Test Engineer
my. // @Kazu_cocoa
my. // KazuCocoa
my. // kazucocoa.wordpress.com
let my = Profile()
6
7.
8. Agenda
✓About our development style
✓ relationship between
departments
✓ Dev process / flow
✓Ice break
✓ Test strategy
✓ Test tools / tips
✓Testing strategy and what
test we run
✓ UI Testing
8
23. What I talked
23
✓ About Cookpad
✓ Our relationship between departments
✓ Our role in iOS development
✓ Our development cycle and process
✓ Who is going to proceed talking each
topics
24. My role and work
~ Focus on technical activities ~
24
28. ✓ Automated tests
✓ running tests, judgement the results
✓ GUI Testing
✓ HTTP Requests check
✓ …
✓ Manual tests
✓ new features
✓ difficult to run automatically
✓ exploratory testing
What we check
in test size L/E
28
29. ✓ Automated tests
✓ running tests, judgement the results
✓ GUI Testing
✓ HTTP Requests check
✓ …
✓ Manual tests
✓ new features
✓ difficult to run automatically
✓ exploratory testing
What we check
in test size L/E
29
30. ~ GUI Test / Scenario Test ~
~ L/E size ~
UI Testing
30
31. ✓ Run tests based on scenarios
✓ We cover over 80% screen transitions
✓ Frequency we run tests
✓ arbitrary timing or before release the
app
Test cases
31
32. ✓ Scenario based automated test
✓ Capture HTTP requests
✓ Screen captures
✓ Image diff
Our automated tests
32
33. ✓ Unexpected broken scenario
✓ Unexpected broken network traffic
✓ Unexpected broken layout
✓ Crashes
What we can find out
33
40. ✓ Easy to figure out what kind of
unexpected defects caused
✓ Regression test
✓ Humans can do other creative activities,
other tasks and so on while running the
automated tests
Advantage of
automated tests
40
41. ✓ If we run all tests with manual
✓ Scenario/GUI diff
✓ It takes around a few hours per a
device at least
✓ We run several devices per release
✓ We can reduce human resources to do
this kind of tasks
Costs
41
42. iOS 8.0 8.1 8.2 8.3 8.4 9.0 9.1 9.2 9.3 10.0
iPho
ne
◯ ◯ ◯ ◯ ◯ ◯ ◯ ◯ -
iPad ◯ ◯ ◯ ◯ ◯ ◯ ◯ ◯ -
iPad
Pro
◯ ◯ ◯ ◯ -
✓ ◯: select devices we run tests
✓ 6~8 devices per a release
✓ iOS8.0 was removed from Xcode
✓ Do test partially with manual
Combination
42
44. ✓ We should update scenario every
release
✓ BTW, other libraries and wrappers
are updated only to solve unstable
environment
✓ We can update scenario within a day
✓ Include libraries and wrappers
✓ We can do other tasks except for
automated test
Maintenance costs
44
45. ✓ Currently, we try convert
Objective-C to Swift
Code changes
45
47. ✓ What is our motivation to enhance
test automation
✓ 2014/1~: I joined to Cookpad
✓ I’m first test engineer
✓Test is not only my activity
✓We’d like to ensure minimum quality
to release the application
Motivation
47
49. ✓ include libraries and scenarios
✓ Appium 0.18.0 ~>
How long have we
developed this tests?
49
50. ✓ We should define concrete plans to
tests
✓ What kind of test should we plan?
✓ We should consider software design,
configuration and so on against
software tests
✓ I used Mindmap to arrange test targets
How to design tests
50
57. ✓ Backend differ from:
✓ Appium 1.6.x
✓ WebDriverAgent for xcuitest strategy
✓ UIAutomator for appium strategy
✓ Before Appium 1.5.x
✓ UIAutomator for appium strategy
✓ Appium try to solve problems between
iOS <=> Appium Server
✓ Users only know about interface
against Appium server
Appium
57
59. ✓ XCUITest supports over iOS9
✓ Cookpad supports iOS8.0
✓ Lifecycle of XCUITest is same as
XCTest
✓ So, we can’t clean environment
every tests.
✓ e.g. Permission dialog
✓ This make tests flaky.
✓ BTW, we should do XCUITest based UI
Testing in the future.
How about XCUITest ?
59