The document summarizes the challenges faced by LINE TODAY in transitioning to continuous integration and automation testing. It discusses moving from a fast track to regular product release, supporting multiple countries/languages. It also covers setting up a CI pipeline with Jenkins, implementing unit, functional, and service monitoring tests with tools like Robot Framework and Selenium. The importance of quality and fixing pain points through retrospectives is emphasized.

1. The Guardian of LINE TODAY
Automation Test
Kent Han

2. • LINE TODAY
• Challenges
• CI and Automation Test
CI Pipeline
Unit Test
Functional Test
Service Monitoring
• Wrap Up
AGENDA

3. LINE TODAY

4. FastTrack RegularTrack
2016.03 - Fast Track as Proof Of Concept
2016.06 - Regular Track as long-term product
GLN - Global LINE News

5. GLN - Global LINE News
TW
18m
TH
36m
IR
3m
ID
32m
JP
58m

6. TW TH ID IR
(RTL)
GLN - Global LINE NewsGLN - Global LINE News

7. Fast Track - 02/26 (TW, TH, ID, US, MM)
Regular Track - 06/16 (TW, TH, ID) 10/04 (IR)
3/2 3/5 3/8 3/113/14 3/17 3/20 3/23 3/263/29 4/1 4/4 4/7 4/10 4/134/16 4/19 4/22 4/25 4/28 5/1 5/4 5/7 5/10 5/13 5/16 5/19 5/225/25 5/28 5/31 6/3 6/6 6/9 6/12 6/15 6/18 6/21 6/24 6/276/30 7/3 7/6 7/9 7/12 7/157/18 7/21 7/24 7/27 7/30 8/2 8/5 8/8 8/11 8/14 8/17 8/208/23 8/26 8/29 9/1 9/4 9/7 9/10
DUB
DPV
Fast
Track
Regular
Track

8. Challenges
Swiftly adjust to survive, or die

9. Feature requests are diverse

10. Agile is the Only Solution

11. How Fast is Your Agile

12. CI + Automation Test

13. Jenkins 2.0
CI Pipeline by Jenkinsews

14. • Jenkins IFTTT LINE Notify
• https://ifttt.com/line
CI Notification

15. Testing Pyramid
https://watirmelon.blog/tag/
testing-pyramid/
Unit test vs. Integration test
• Speed
• Stable
• Easy to troubleshoot
Testing Pyramid

16. Pull Request must be checked by unit test
Unit Test

17. Static Code Analysis
• Checkstyle
• FindBugs
• PMD
• Duplicate Code

18. Testing Tools
• Python
• Robot Framework
• Selenium
• Appium
• Jenkins
• Gatling
Functional Test

19. Why Did We Adopt
Robot Framework?
• Human-readable test cases for
Maintainability
• Python and Java support for Versatility
• --rerunfailed for flaky tests
*** Test Cases ***
Add A Comment To An Article
[Tags] comment RAT
New An Article In Category
Add Comment To The Article 1 Times
Comment Count Of The Article Is 1
Reply A Comment To A Comment
[Tags] reply RAT
New An Article In Category
Add Comment To The Article 1 Times
Comment Count Of The Article Is 1
Reply To The Comment With Text Comment
Comment Should Be Included In Article
Vote Up To A Comment
[Tags] vote RAT
New An Article In Category
Add Comment To The Article 1 Times
Comment Count Of The Article Is 1
Vote The Comment With UP

20. Robot Framework Report

21. Content Working Flow

22. Feeding Test
• Average 5,500 articles
imported a day
• Should ensure the input
stream
• Fake articles for testing
Thumbnail
Image
Video
Youtube
<?xml version='1.0' encoding='UTF-8'?>
<articles>
<UUID>c11ae751-b180-11e6-96c0-985aeb8c0520</UUID>
<time>1461117682767</time>
<article>
<ID>c11b1973-b180-11e6-8d03-985aeb8c0520</ID>
<nativeCountry>TW</nativeCountry>
<language>zh</language>
<startYmdtUnix>1479907723508</startYmdtUnix>
<endYmdtUnix>1571117682767</endYmdtUnix>
<title>20161123212843_all_medias</title>
<category> </category>
<publishTimeUnix>1479907723508</publishTimeUnix>
<contents>
<image>
<description>Mountain is beautiful!</description>
<url>yama_1.png</url>
</image>
<video>
<description>Kobe Top 1</description>
<thumbnail>thumbnail_1.jpg</thumbnail>
<url>kobe_1.mp4</url>
</video>
<video>
<description>Youtube</description>
<url>https://www.youtube.com/embed/c7rCyll5AeY</url>
</video>
<text>
<content> Google Google
Alphabet
Larry Page Alphabet Google Sundar Pichi
Google Larry Page
Google ......
Alphabet
</content>
</text>
</contents>
</article>
</articles>

23. CMS Test
• OA Digest
• Timeline Digest
• Category publication
• When new features are added,
basic functions should not fail

24. Front-end Test
• Selenium covers the most
• Appium for the mobile-only feature
• API test as a tool to generate testing
data

25. • Internal tools
iMON - application error
NSight - system error
• Health check script
Web response time
RabbitMQ management api
ElasticSearch cluster/node/stats/
status api
Service Monitoring

26. 0
100
200
300
400
v2.1 v2.2 v2.3 v3.0 v3.1 v3.2 v3.3
major bugs total bugs
Bug Stats

27. Wrap Up
Quality is everyone’s KPI
Find a pain point in retrospective then improve in next sprint

28. THANK YOU