This document introduces a managed Kubernetes as a service (KAAS) that provides concise summaries in 3 sentences or less that provide the high level and essential information from the document. The KAAS addresses problems with different tooling, versions, and configurations across clusters by providing a standardized Kubernetes platform. It aims to reduce operation costs and improve quality by ensuring high availability, performance optimization, and private cloud integration. The KAAS leverages Rancher for declarative operations and integrates custom controllers to enable load balancing, persistent storage, and other private cloud services within Kubernetes clusters.

2. INTRODUCTION OF
MANAGED K8S SERVICE

3. KAAS IN LINE PRIVATE CLOUD

4. BEFORE KAAS
VM
Kubernetes Kubernetes
Container
Container
Container
Container
Container
Container
Container
Container
Container
Container
Container
Container
Developers A in Japan Developers B in Taiwan
Private Cloud
Private Cloud Developers
Application Developer
OS
VM
OS
VM
OS
BM
OS
BM
OS
BM
OS
IaaS
Responsibility border
・・・・・・

5. ● Different Toolset
○ Different Version
○ Different Add-on
○ Different Plugin
● Distributing Knowledge
PROBLEMS WE FACED
● Large Operation Cost
○ Deploy Cluster
○ Extend Cluster
○ Troubleshooting Cluster
● Quality of Cluster
○ No HA Configuration
○ No Performance Consideration
○ No Private Cloud Collaboration
From Application Developer point of view From Company-wide point of view

6. OUR MISSION
For more than 2000 developers
( 100+ clusters )
KaaS
Kubernetes
OperatorKubernetes Cluster
Performance
High Availability
Private Cloud Collaboration

7. PERFORMANCE CONSIDERATION
Kubernetes
Operator
Kubernetes Cluster
Performance
High Availability
Private Cloud Collaboration

8. HOW USER BUILD CONTAINER NETWORK
VM Network (L3 Reachability ensured)
Kubernetes Cluster
VM VM VM
vxlan
Container Network
Private Cloud Agnostic
Encapsulate Overhead

9. DELEGATE CONTAINER NETWORK TO CLOUD
VM Network (L3 Reachability ensured)
Kubernetes Cluster
vxlan
Container Network
Less encapsulate
overhead
Build container network
with performance optimization
Tenant Network For Container (L2 Reachability ensured)
VM VM VM host-gw
VXLAN Offload

10. HIGH AVAILABILITY CONSIDERATION
Kubernetes
Operator
Kubernetes Cluster
Performance
High Availability
Private Cloud Collaboration

11. etcd etcd etcd
3•etcd
1
TOLERATE 1 NODE FAILURE DESIGN
controller controller controller
3
•kube-apiserver
•kube-controller-manager
•kube-scheduler
•kubelet
•kube-proxy
× ×
×
2
worker worker worker
N
•kubelet
•kube-proxy -
Toleration LimitOur Deployment

12. SELF HEALING BY REPLACING NODE
etcd etcd etcd
controller controller controller
worker worker worker
× ×
× etcd
controller controller
When we detect VM shutdown/error in Private Cloud,
Unavailable node will be deleted from cluster and
new node will be added when the number of unavailable node is
lower than toleration limit
Delete
Delete Delete
New
New New

13. PERIODICALLY BACKUP OF ETCD
Snapshot of etcd
Keep at least 2 days
etcd etcd etcd
14:30
14:45
etcd etcd etcd
Snapshot as of 14:30
14:40
etcd etcd etcd× ××
Object Storage
New New New
Kubernetes Cluster
Kubernetes Cluster
Kubernetes Cluster

14. PRIVATE CLOUD COLLABORATION
Kubernetes
Operator
Kubernetes Cluster
Performance
High Availability
Private Cloud Collaboration

15. PERSISTENT VOLUME SUPPORT
csi-cinder-provisioner
csi-attacher-cinderplugin
csi-nodeplugin-
cinderplugin
csi-nodeplugin-
cinderplugin
csi-nodeplugin-
cinderplugin
kind: StorageClass
metadata:
name: cinder-ssd
provisioner: csi-cinderplugin
kind: PersistentVolumeClaim
spec:
storageClassName: cinder-ssd
resources:
requests:
storage: 1G
Verified Add-on: Cinder Standalone Persistent Volume Provider
Kubernetes Cluster

16. TYPE LOADBALANCER SUPPORT
Custom Controller for Type Loadbalancer
In-house Loadbalancer
kind: Service
spec:
selector:
app: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80
nodePort: 30000
type: LoadBalancer
Create New VIP
Add Real Server
Develop Add-on: Custom Controller for in-house LB
Kubernetes Cluster

17. PRIVATE CLOUD HAVE VARIOUS SERVICE
DNS
We provide
useful managed service
Pod
Pod
We want to use
We want to create record
Kubernetes Cluster
Service

18. WHEN USER WANT TO USE ...
DNS
Pod
Pod
Create/Configure
Pass access information
Kubernetes Cluster

19. CONFIGURE OTHER SERVICE VIA K8S
DNS
Pod
Pod
kind: Redis
spec:
mode: sentinel
node: normal
kind: Service
====
Redis Instance
====
kind: Secret
====
Redis Credential
====
Kubernetes Cluster
Develop Add-on: Custom Controller for Managed Service
Custom Controller for Managed Service

20. HOW WE OPERATE K8S CLUSTER
Kubernetes
OperatorKubernetes Cluster
Performance
High Availability
Private Cloud Collaboration

21. OVERVIEW OF KAAS
Cluster Operation
- Cluster Create
- Cluster Update
- Add Worker
Use Cluster
- Deploy application
- Scale application
Manage Cluster
- Deploy - Monitor
- Update
Private Cloud User
Automate Operating
Multiple Cluster
API
Kubernetes
Kubernetes
Kubernetes

22. WHY WE USE RANCHER?
● Use attractive ecosystem of Kubernetes
- Declarative Operation
- Leader election
- Data Store (CRD)
- Detect Change of Data
● Easy to focus on own business logic
● Easy to implement asynchronous logic
● Retry logic implemented by design
Merits from Operator Pattern
● OSS
● Many Active Developers
● Private Cloud Agnostic
● Adopt Operator Pattern
Why Rancher?Why Rancher? Merits from Operator Pattern

23. AFTER K8S AS A SERVICE

24. • Users can use k8s without strong knowledge
• Users are completely free from operating Kubernetes Cluster
• Users can make use of our Private Cloud from k8s easily
ACCELERATE APPLICATION
DEVELOPMENT

25. THANK YOU