This document discusses testing practices at Cookpad, a company that develops large web and mobile applications. It outlines their philosophy of prioritizing user experience and keeping development cycles quick through practices like frequent small releases and automated testing. For their web application, which is a large Ruby on Rails monocodebase, it describes their extensive use of RSpec tests as well as tools they developed to help speed up testing. For their mobile applications, it discusses their bi-weekly release process and focus on unit, integration and UI testing using tools like Appium and Espresso. It also acknowledges challenges in moving to microservices and continuous testing of mobile apps.

2. Name: Matsuo Kazuaki
Twitter: @Kazu_cocoa
Job:(Mobile)Test Engineer
@Cookpad

3. index
• Our philosophy
• Development - Web applications
• Development - Mobile applications
• Conclusion

4. •
•
•
•
•
•
•
•
•
•
•
•
Our Manifesto

5. User First
Our Philosophy

6. Our
Huge Rails Application

7. Web Application
(include API Server)

8. Web Site

9. % rake stats
+----------------------+-------+-------+---------+---------+-----+-------+
| Name | Lines | LOC | Classes | Methods | M/C | LOC/M |
+----------------------+-------+-------+---------+---------+-----+-------+
| Controllers | 49708 | 40012 | 537 | 4054 | 7 | 7 |
| Helpers | 15122 | 12420 | 15 | 1432 | 95 | 6 |
| Models | 98535 | 77593 | 1786 | 8670 | 4 | 6 |
| Mailers | 2122 | 1698 | 41 | 198 | 4 | 6 |
| Workers | 639 | 540 | 20 | 31 | 1 | 15 |
| Chanko units | 12544 | 10354 | 6 | 255 | 42 | 38 |
| Libraries | 49927 | 41289 | 603 | 3724 | 6 | 9 |
| Feature specs | 45798 | 37707 | 0 | 208 | 0 | 179 |
| Request specs | 39112 | 33563 | 0 | 16 | 0 | 2095 |
| Routing specs | 664 | 536 | 0 | 0 | 0 | 0 |
| Controller specs | 60829 | 50293 | 7 | 127 | 18 | 394 |
| Helper specs | 79735 | 65835 | 5 | 73 | 14 | 899 |
| Model specs | 151719|125330 | 6 | 128 | 21 | 977 |
| Worker specs | 862 | 715 | 0 | 1 | 0 | 713 |
| Chanko unit specs | 10396 | 8379 | 0 | 9 | 0 | 929 |
| Library specs | 23528 | 19679 | 27 | 128 | 4 | 151 |
+----------------------+-------+-------+---------+---------+-----+-------+
| Total |641240 |525943 | 3053 | 19054 | 6 | 25 |
+----------------------+-------+-------+---------+---------+-----+-------+
Code LOC: 183906 Test LOC: 342037 Code to Test Ratio: 1:1.9

10. % bundle show | wc -l
284

11. Deployment PipelinePR
merge
pull
tag
deploy
deploy

12. • Frequent small release
• 10+ deploy per a day
• To decrease incident risk per deploy
• Minimise period of incidents
• Roll back with low risk
• Finish rollback within 1 min
Release cycle

13. Development style
• Team exist in each departments
• Basically, they don’t have any test
engineers.
• One monolithic application
• Challenging to Microservices…
• Each developers are in charge of their
own implemented feature.

15. Must write test code
• All developers should implement test
code before merge pr to master.
• Why…
• To keep development cycle
• We can’t estimate side-effect

16. Automated Test
• Must implement specs when implement features
• RSpec based tests
• 20000+ RSpec examples
• It takes many hours finishing all test.
• Capybara
• with Headless browser

17. Hand Test
• Checked on production test
• Before release
• Checked feature by developers
• Tests on development phase
• User test(Internal benchmark)
• Security Test
• and so on…

18. Priority in test code
• Keep test quick
• We want to keep release cycle quick.
• Developed RRRSpec to keep speed
• Finish all spec test cases around
5 hour => Finish within 10 minutes

19. Tools developed by Cookpad
• Chanko
• protect incidents
• RRRSpec
• Distributed Rspec framework
• Switch_point
• change target DB
• Database_rewinder
• fast database cleaner
• Kage
• proxy to send copied real request to test
environments.

20. Challenge to Microservices
• Cookpad is a huge monolithic
application

21. Challenge to Microservices
• Many small web applications
• Keep release cycle quickly
• Stable and quick deploy
• ex. mamiya(our original tool)

22. How to keep stable them?
• Dependency
• Distributed Computing…
It’s one of my task… 😢

23. Mobile Application
(iOS/Android)

24. Release flow

25. PipelinePR
merge
pull
tag
publish
tointernal

26. • Around 2 week release cycle
• Decide which feature implement in
which cycle by each departments
• High release cycle lead
unpleasant
• Optimize to Google Play / App Store
Release

27. Development
• mobile-infra
• iOS/Android engineers: 3
• Test Engineer: 1 <= ME!!✊
• Each departments
• Android / iOS engineers
• API engineers (also develop web site)
• Not one team develop our application.

28. Test Level
• Unit Test
• Integration Test
• Feature Test
• Performance / GUI and so on.
• Make more efficient for tests

29. Develop/Test Tools
• Debug
• Stetho, PonyDebugger
• Enhance Testability
• AndroidJUnit4 and Java libraries
• XCTest, Mocks
• Appium, Espresso, UIAutomation
• Reporting
• Turnip / turnip_formatter
• kobold

30. Challenge to continuous
stable release…
• Reduce review load
• Dokumi
• Comment the result of static
analytics automatically.
• Enhance testability
• DI into Android…
• Establish hermetic environment

31. Other tests

32. • Performance
• Security
• Attractive
and so on…

33. Conclusion
• We test to keep high speed
development cycle.
• We challenge to Microservices.
• We challenge how to value mobile
applications.

34. Thank you.