This document discusses deploying WSO2 middleware on Kubernetes. It provides an overview of Kubernetes architecture and components, and how various Kubernetes features like pods, replication controllers, services, and overlay networking are used. It also describes WSO2 Docker images, Carbon reference architectures for Kubernetes, and the deployment workflow. Monitoring of Kubernetes cluster health using tools like cAdvisor, Heapster, Grafana and InfluxDB is also covered briefly.

1. Deploying WSO2 Middleware
on Kubernetes
Imesh Gunaratne
Senior Technical Lead, WSO2

2. Agenda
● PaaS Solutions for WSO2 Middleware
● Deploying WSO2 Middleware on Kubernetes
○ Kubernetes Architecture
○ Kubernetes Features Used
○ WSO2 Docker Images
○ WSO2 Carbon Reference Architectures for Kubernetes
○ Deployment Workflow
○ WSO2 Kubernetes Artifacts
○ Monitoring

3. PaaS Solutions for
Middleware

4. PaaS Solutions for WSO2 Middleware

5. Core PaaS Features Needed for a
Production Deployment
● Auto healing
● Autoscaling
● Automatic cluster discovery
● Dynamic load balancing
● VM/Container support
● Multi-tenancy
● Configuration orchestration
● Artifact distribution
● Multi-region/cloud deployments
● Centralized logging
● Monitoring
● Metering

6. Deploying WSO2 Middleware
on Kubernetes

7. An Introduction to Kubernetes
● An open source container cluster management system
● Started by Google with decade and half experience on
running containers at scale (Borg)
● Written in Go
● Can be deployed on any Infrastructure/IaaS
● Over 600 contributors
http://kubernetes.io/v1.0/docs/whatisk8s.html

8. Kubernetes High Level Architecture
Node1 Node2 Node n
Physical Network
Master
Overlay Network (Flannel/OpenVSwitch/Weave)APIServer
Scheduler
ControllerManager
etcd

9. Kubernetes Component Architecture

10. Kubernetes Features used by WSO2
Middleware
● Pods
● Replication Controllers (RC)
● Horizontal Pod Autoscalers (HPA)
● Services
● Ingress/Service Load Balancer
● Overlay Network
● Service Accounts & Secrets
● Namespaces

11. Kubernetes Pods
● A group of containers which can share
resources and context
● Shared namespaces:
○ PID namespace (processes visible globally)
○ network namespace (same IP and port
space)
○ IPC namespace (SystemV IPC or POSIX)
○ UTS namespace (share a hostname)
http://kubernetes.io/v1.0/docs/user-guide/pods.html
Pod
C1 C2 C3

12. Kubernetes Replication Controllers
● Provides features for replicating pods:
○ Auto-healing
○ Manual scaling
○ Rolling updates
○ Multiple release tracks
http://kubernetes.io/v1.0/docs/user-guide/replication-controller.html
Pod
Template
Replicas = n
Pod 1 Pod 2 Pod n
Replication
Controller

13. Kubernetes Horizontal Pod Autoscaler
● Provides features for horizontally
autoscaling pods
○ CPU consumption based
○ Currently in Alpha
http://kubernetes.io/v1.1/docs/user-guide/horizontal-pod-autoscaler.html
Pod
Template
Replicas = n
Pod 1 Pod 2 Pod n
Replication
Controller
Horizontal Pod
Autoscaler

14. ● Service types:
○ ClusterIP (default)
○ NodePort
○ LoadBalancer
● Service discovery:
○ DNS
○ Environment variables
● Client IP based Session Affinity
http://kubernetes.io/v1.0/docs/user-guide/services.html
Kubernetes Services
Node IP: 172.17.8.102
Port: 9443
Domain Name: service1
IP: 10.2.10.20
Port: 9443
NodePort: 32001
Protocol: TCP
Pod 1 Pod 2 Pod n
Service
L1
L1 L1 L1
Node

15. ● Unique IP addresses
● Provides pod to pod networking
● Options available:
○ Flannel (default)
○ OpenVSwitch
○ Weave
○ Calico
http://kubernetes.io/v1.1/docs/admin/networking.html
Kubernetes Overlay Network

16. WSO2 Docker Images
https://github.com/wso2/dockerfiles

17. WSO2 Carbon Reference Architecture 1
Pod 1 Pod 2
Manager Cluster
Services
Pod 3 Pod 4 Pod n
Worker Cluster
Services
Manager
Replication
Controller
Worker Replication
Controller
Client
Gov
Reg
Conf
Reg
User
Store

18. WSO2 Carbon Reference Architecture 2
Pod 1
Gateway Mgr
Services
Pod 2 Pod 3
Gateway Worker
Services
Client
Pod 4 Pod 5
Key Manager
Services
Pod 6 Pod 7
Store Services
Gateway
Mgr RC
Gateway
Worker RC
Key
Manager RC
Store RCGov
Reg
Conf
Reg
User
Store
API-M
DB
A volume mount for
synchronizing API
artifacts
Pod 8 Pod 9
Publisher Services
Publisher
RC

19. Carbon Cluster Discovery on Kubernetes
Pod 1 Pod 2 Pod n
Service
Server 1 Server 2 Server n
Pod 1
Service
Server 1
Pod 1 Pod 2
Service
Server 1 Server 2
Hazelcast member
initialization
Hazelcast member
initialization
Hazelcast member
initialization
1 2 n

20. Deployment Workflow
3
1 2
Build Docker
Image
Import Docker Image
to Docker Registry
Application Artifacts
Kubernetes
Membership
Scheme
Product Distribution
Docker Image (DI)
Puppet Module
+ Hiera Files
JDK Deploy Kubernetes
Artifacts
Kubernetes
Artifacts

21. WSO2 Kubernetes Artifacts
● WSO2 Kubernetes Artifacts Git Repository
○ Kubernetes Membership Scheme
○ Kubernetes Services
○ Kubernetes Replication Controllers
https://github.com/wso2/kubernetes-artifacts

22. Monitoring Kubernetes Cluster Health

23. Monitoring Kubernetes Cluster/Node/Pod
Health
cAdvisorcAdvisor
Node 1
cAdvisor
Heapster
Node 2 Node n
Grafana
InfluxDB

24. Thank you!