https://jenkins.jp/juc2018/ How to modernize legacy Jenkins pipeline with useful plugins. Migrate from Freestyle to Pipeline. Provide scalable pipeline with Multibranch Pipeline.

1. Modernize deployment pipeline
Sep 23, 2018
Takizawa Takeshi
Ecosystem Services Dept.
Rakuten, Inc.

2. 2
Agenda
• Self introduction
• Rakuten services
• Membership Section
• Issues
• Solutions
• Results

3. 3
Self introduction
Name: Takeshi Takizawa
GitHub ID: TakiTake
Company: Rakuten, Inc.
Role: DevOps Team Leader
(Jenkins Ojisan)

4. 4
Rakuten services
Serving users worldwide through businesses based in 29 countries and regions.
Our major services available globally. (09/23/2018)
https://global.rakuten.com/corp/about/

5. 5
Membership section’s role and responsibility
Rakuten Group services have more than 1.2 billion members.
Membership provides Identity and Access Management (IAM) functions for Rakuten Group.
Rakuten Ichiba login page (09/20/2018)

6. 6
Membership section’s role and responsibility
Rakuten Group services have more than 1.2 billion members.
Membership provides Identity and Access Management (IAM) functions for Rakuten Group.
Rakuten Ichiba top page (09/20/2018)

7. 7
Platform
Migrating from Physical and VM to the Container (Kubernetes)
Most of the servers are existing in Rakuten Datacenter.
Migrating
Physical / VM
Container

8. 8
Deployment Pipeline
Chain Jenkins jobs with Parameterized Trigger Plugin.
Over 600 deployment pipelines are existing.

9. 9
Chain Jenkins jobs with Parameterized Trigger Plugin.
Over 600 deployment pipelines are existing.
Build APP
Prepare
config files
Build
Docker
container
Analyze
vulnerability
Deploy to the Kubernetes
Deployment Pipeline

10. 10
Deployment Pipeline
Different Jobs per application
Same # as (application * ENV)
Chain Jenkins jobs with Parameterized Trigger Plugin.
Over 600 deployment pipelines are existing.

11. 11
Deployment Pipeline
Common Job
Only 1 Job
Chain Jenkins jobs with Parameterized Trigger Plugin.
Over 600 deployment pipelines are existing.

12. 12
Environment
Approximately 100 DEV and QA ENGs share the fixed environments.
Developer1
ENV1
ENV2
ENV3
Developer2
Developer3
QA1

13. 13
Issue 1
Maintenance cost of the pipeline is high.
Jenkins Jobs are tightly coupled to each other

14. 14
Issue 1
Maintenance cost of the pipeline is high.
Hard to modify Job config via UI one by one

15. 15
Solution 1
Migrate Jenkins Job from Freestyle to Pipeline.

16. 16
Solution 1 – Step 1
Migrate Jenkins Job from Freestyle to Pipeline.
build_app.sh
Move hardcoded steps from Job to script file.

17. 17
Solution 1 – Step 2
Call scripted steps from wrapper function.
Register the functions as a Global Pipeline Library.
vars/buildApp.groovy

18. 18
Solution 1 – Step 3
Call wrapper function from Jenkinsfile which existing in each APP repository.
Jenkinsfile (sample)

19. 19
Issue 2
Lack of the environment for Development and QA.
Developer1
ENV1
ENV2
ENV3
Developer2
Developer3
QA1

20. 20
Issue 2
Lack of the environment for Development and QA.
Developer1
ENV1
ENV2
ENV3
Developer2
Developer3
QA1
Waiting!!

21. 21
Solution 2
Control environment creation with Multibranch Pipeline.

22. 22
Solution 2 – Step 1
Create new branch then new pipeline is also created automatically.
Developer/QA
my-branch ENV
feature/abc ENV
hotfix/xyz ENV
my-branch

23. 23
Solution 2 – Step 2
Use wildcard DNS record, e.g. *.id.rakuten.co.jp.
Developer/QA
DNS Server
Ingress
(Reverse Proxy)
→ my-branch.id.rakuten.co.jp
← Ingress IP
1.
2. my-branch.id.rakuten.co.jp

24. 24
Solution 2 – Step 3
Use domain mapping in the Ingress.
Developer/QA
DNS Server
Ingress
(Reverse Proxy)
my-branch ENV
feature/abc ENV
hotfix/xyz ENV
1.
2.
3. my-branch.id.rakuten.co.jp

25. 25
Results
• Maintenance cost of the pipeline is high
=> Pipeline as Code
• Lack of the environment for Development and QA
=> Multibranch Pipeline
Tip: You’d better concentrate on controlling the flow to keep portability.
Tip: You’d better delete the ENV daily or weekly to use server resource efficiently.

26. 26
Thank you!
takeshi.takizawa@rakuten.com