Building Portable Applications with
Kubernetes
Oleg Chunikhin |CTO, Kublr
Introductions
Oleg Chunikhin
CTO, Kublr
 20+ years in software architecture & development
 Working w/ Kubernetes since its release in 2015
 CTO at Kublr—an enterprise ready container
management platform
 Twitter @olgch; @kublr
Like what you hear? Tweet at us!
Reasons for Portability
• Move load quickly (geography, cost, features)
• Lifecycle (dev/test/staging/production)
• Disaster recovery
• Split-tier architecture (application tiers may reside
in different environments)
• Cloud-bursting
@olgch; @kublr
Best Practices for Portable Applications
12-factor app is your bible – especially verses 2, 3, 4, 6, 7, 12
2. Explicitly declare and isolate dependencies
3. Store configuration in the environment
4. Treat backing services as attached resources
6. Execute the app as one or more stateless processes
7. Export services via port binding
12. Run admin/management tasks as one-off processes
@olgch; @kublr
Hard Problems
Ingress traffic
• Routing
• HTTPS/SSL certificates
State persistence
• Shared file storage
• Reliable block storage
• Database
Messaging
@olgch; @kublr
Application Needs
Application
Database Exclusive FS Shared FS
Routing SSL TermBalancing
@olgch; @kublr
“Rich” Platform
Application
Database Exclusive FS Shared FS
Routing SSL TermBalancingPlatform
Platform
@olgch; @kublr
Not Quite as Rich of a Platform
Application
? Database ? Exclusive FS Shared FS
? Routing ? ? SSL Term ?Balancing
Platform
Platform
@olgch; @kublr
“Poor” Platform
Application
? Database ? Exclusive FS ? Shared FS?
? Routing ? ? SSL Term ?? Balancing ?
Platform@olgch; @kublr
Kubernetes to the Rescue
• Extreme ease and flexibility of component configurations and connections
• Configuration templating tools
• Helm
• Abstractions and extensible framework for ingress traffic processing
• Service
• Ingress
• Ingress Controllers
• Abstractions and extensible framework for storage management
• Volumes
• Persistent Volumes
@olgch; @kublr
Service
External node ports or external load balancer
Kubernetes cluster
Pod A-1
10.0.0.3
Pod A-2
10.0.1.5
Pod B-1
10.0.0.8
SrvB
10.7.0.3
Internal service
SrvA
10.7.0.1
SrvC
10.7.0.5
Ext
Resource
@olgch; @kublr
Ingress
Kubernetes cluster
SrvA
10.7.0.1
SrvB
10.7.0.3
Ingress rule 1:
abc.com/abc > SrvA
Ingress rule 2:
def.com/def > SrvB
Pod A-1
Pod A-2
Pod B-1
@olgch; @kublr
Ingress Controller
Edge / entry point / load balancer
Kubernetes cluster
SrvA
10.7.0.1
SrvB
10.7.0.3
Ingress
controller
Ingress rule 1:
abc.com/abc > SrvA
Ingress rule 2:
def.com/def > SrvB
Pod A-1
Pod A-2
Pod B-1
@olgch; @kublr
Configurability and Persistence
Pod
Container 1 Container 2
Persistent Volume:
NFS, Gluster, ceph,
EBS, dir, etc
Volume
Volume
Claim
Volume
Mount
Volume
Mount
Config Map
Secret
Storage Class
Static or dynamically allocated
Demo Application
WordPress
MySql Shared FS MySql Data
Routing SSL TermBalancing
@olgch; @kublr
Demo Application Helm Package
WordPress Deployment
MySql PVC
MySql Service
WordPress Ingress
WordPress Service
MySql Deployment
@olgch; @kublr
Demo Environment
AWS Kubernetes
Master
Ubuntu
Worker 1
Ubuntu
Worker 2
Ubuntu
Worker 3
Ubuntu
Azure Kubernetes
Master
RHEL
Worker 1
RHEL
Worker 2
RHEL
Worker 3
RHEL
kubectl --context=aws ...
helm --kube-context=aws ...
kubectl --context=azure ...
helm --kube-context=azure ...
@olgch; @kublr
Demo Application – POC / Evaluation (Azure)
WordPress
MySql
Routing SSL TermBalancing
MySql Data emptyDir Shared FS
ephemeral storage
provided by Kubernetes
Accessible from inside the cluster only via HTTP
@olgch; @kublr
Demo Application – POC / LB (Azure)
WordPress
MySql
Routing SSL Term
MySql Data emptyDir Shared FS
ephemeral storage
provided by Kubernetes
Accessible from inside the cluster only via HTTP
@olgch; @kublr
Azure LB
WordPress
MySql MySql Data emptyDir
Ingress Routing
Ingress SSL
Term + LEGO
ELB Balancing
Shared FS
ephemeral storage
provided by Kubernetes
Accessible via HTTPS on ELB with the given host
Demo Application – POC / Ingress
@olgch; @kublr
Demo Application – Production – RDS
AWS
WordPress
AWS RDS MySql
Ingress Routing
Ingress SSL
Term + LEGO
ELB Balancing
AWS EBS AWS EFS
persistent storage on AWS RDS
allocated outside of Kubernetes
Accessible via HTTPS on ELB with the given host
WordPress
MySql
Ingress Routing
Ingress SSL
Term + LEGO
ELB
Rook operator and Ceph cluster
MySql Data on Ceph
replica pool Ceph File System
Accessible via HTTPS on ELB with the given host
persistent storage on self-hosted
MySql and Ceph
Demo Application – Prod – Ceph data – AWS
Data pool
mon
config
data
config
data
monmon
config
data
Cloud Native Storage: Rook/Ceph
raw data
osd
raw data
osd
raw data
mdsosd
Data pool
Image Image
Ceph
Filesystem
Components
Abstractions
Ceph
rgw
S3/Swift
Object Store
mgr
Rook
Operator
CSI plugins
osdosdganesha
NFS
CephCluster
Block Pool
Object Store
Filesystem
NFS
Object Store User
Provisioners
rbd-mirror
@olgch; @kublr
Managed / Self-Hosted Options
File system: AWS EFS; Rook; OpenEBS; GlusterFS
Database: AWS RDS; MySql; PostgreSQL
Ingress: AWS ELB/ALB/NLB; Ingress controller
Certificates: AWS Certs; Let’s Encrypt; manual; self-signed
Messaging: AWS SQS; RabbitMQ; ActiveMQ
Azure has equivalent managed services for most app needs
@olgch; @kublr
Gotchas
• Self-hosted is more difficult to
operate than managed
• Different implementations
have varying functionalities
and QoS
• Performance
• Standards compliance
For example
• AWS EBS is AZ local
• Let’s Encrypt limits certificate
issuance rate
• Managed services may be better
hardware tuned
• Self-hosted services may be
better application tuned
@olgch; @kublr
Takeaways
• Cloud native Kubernetes applications are
portable, and easy to test, experiment,
and configure
• Portability tools
• Helm configuration templating
• Kubernetes abstractions: PV, PVC,
Ingress, Service etc
• Using self-hosted resources where
managed are not available
• Managed vs self-hosted services
considerations
• Different platforms
• On AWS and Azure
• On Ubuntu and RHEL
• Different ingress options
• With and without reverse-proxy
• With and without SSL
• Different persistence options
• Ephemeral storage
• Managed database
• Managed block storage, self-hosted
database
• Self-hosted cloud native storage
@olgch; @kublr
Q&A
Oleg Chunikhin
CTO
oleg@kublr.com
Kublr | kublr.com
Thank you!

Building Portable Applications with Kubernetes

  • 1.
    Building Portable Applicationswith Kubernetes Oleg Chunikhin |CTO, Kublr
  • 2.
    Introductions Oleg Chunikhin CTO, Kublr 20+ years in software architecture & development  Working w/ Kubernetes since its release in 2015  CTO at Kublr—an enterprise ready container management platform  Twitter @olgch; @kublr Like what you hear? Tweet at us!
  • 3.
    Reasons for Portability •Move load quickly (geography, cost, features) • Lifecycle (dev/test/staging/production) • Disaster recovery • Split-tier architecture (application tiers may reside in different environments) • Cloud-bursting @olgch; @kublr
  • 4.
    Best Practices forPortable Applications 12-factor app is your bible – especially verses 2, 3, 4, 6, 7, 12 2. Explicitly declare and isolate dependencies 3. Store configuration in the environment 4. Treat backing services as attached resources 6. Execute the app as one or more stateless processes 7. Export services via port binding 12. Run admin/management tasks as one-off processes @olgch; @kublr
  • 5.
    Hard Problems Ingress traffic •Routing • HTTPS/SSL certificates State persistence • Shared file storage • Reliable block storage • Database Messaging @olgch; @kublr
  • 6.
    Application Needs Application Database ExclusiveFS Shared FS Routing SSL TermBalancing @olgch; @kublr
  • 7.
    “Rich” Platform Application Database ExclusiveFS Shared FS Routing SSL TermBalancingPlatform Platform @olgch; @kublr
  • 8.
    Not Quite asRich of a Platform Application ? Database ? Exclusive FS Shared FS ? Routing ? ? SSL Term ?Balancing Platform Platform @olgch; @kublr
  • 9.
    “Poor” Platform Application ? Database? Exclusive FS ? Shared FS? ? Routing ? ? SSL Term ?? Balancing ? Platform@olgch; @kublr
  • 10.
    Kubernetes to theRescue • Extreme ease and flexibility of component configurations and connections • Configuration templating tools • Helm • Abstractions and extensible framework for ingress traffic processing • Service • Ingress • Ingress Controllers • Abstractions and extensible framework for storage management • Volumes • Persistent Volumes @olgch; @kublr
  • 11.
    Service External node portsor external load balancer Kubernetes cluster Pod A-1 10.0.0.3 Pod A-2 10.0.1.5 Pod B-1 10.0.0.8 SrvB 10.7.0.3 Internal service SrvA 10.7.0.1 SrvC 10.7.0.5 Ext Resource @olgch; @kublr
  • 12.
    Ingress Kubernetes cluster SrvA 10.7.0.1 SrvB 10.7.0.3 Ingress rule1: abc.com/abc > SrvA Ingress rule 2: def.com/def > SrvB Pod A-1 Pod A-2 Pod B-1 @olgch; @kublr
  • 13.
    Ingress Controller Edge /entry point / load balancer Kubernetes cluster SrvA 10.7.0.1 SrvB 10.7.0.3 Ingress controller Ingress rule 1: abc.com/abc > SrvA Ingress rule 2: def.com/def > SrvB Pod A-1 Pod A-2 Pod B-1 @olgch; @kublr
  • 14.
    Configurability and Persistence Pod Container1 Container 2 Persistent Volume: NFS, Gluster, ceph, EBS, dir, etc Volume Volume Claim Volume Mount Volume Mount Config Map Secret Storage Class Static or dynamically allocated
  • 15.
    Demo Application WordPress MySql SharedFS MySql Data Routing SSL TermBalancing @olgch; @kublr
  • 16.
    Demo Application HelmPackage WordPress Deployment MySql PVC MySql Service WordPress Ingress WordPress Service MySql Deployment @olgch; @kublr
  • 17.
    Demo Environment AWS Kubernetes Master Ubuntu Worker1 Ubuntu Worker 2 Ubuntu Worker 3 Ubuntu Azure Kubernetes Master RHEL Worker 1 RHEL Worker 2 RHEL Worker 3 RHEL kubectl --context=aws ... helm --kube-context=aws ... kubectl --context=azure ... helm --kube-context=azure ... @olgch; @kublr
  • 18.
    Demo Application –POC / Evaluation (Azure) WordPress MySql Routing SSL TermBalancing MySql Data emptyDir Shared FS ephemeral storage provided by Kubernetes Accessible from inside the cluster only via HTTP @olgch; @kublr
  • 19.
    Demo Application –POC / LB (Azure) WordPress MySql Routing SSL Term MySql Data emptyDir Shared FS ephemeral storage provided by Kubernetes Accessible from inside the cluster only via HTTP @olgch; @kublr Azure LB
  • 20.
    WordPress MySql MySql DataemptyDir Ingress Routing Ingress SSL Term + LEGO ELB Balancing Shared FS ephemeral storage provided by Kubernetes Accessible via HTTPS on ELB with the given host Demo Application – POC / Ingress @olgch; @kublr
  • 21.
    Demo Application –Production – RDS AWS WordPress AWS RDS MySql Ingress Routing Ingress SSL Term + LEGO ELB Balancing AWS EBS AWS EFS persistent storage on AWS RDS allocated outside of Kubernetes Accessible via HTTPS on ELB with the given host
  • 22.
    WordPress MySql Ingress Routing Ingress SSL Term+ LEGO ELB Rook operator and Ceph cluster MySql Data on Ceph replica pool Ceph File System Accessible via HTTPS on ELB with the given host persistent storage on self-hosted MySql and Ceph Demo Application – Prod – Ceph data – AWS
  • 23.
    Data pool mon config data config data monmon config data Cloud NativeStorage: Rook/Ceph raw data osd raw data osd raw data mdsosd Data pool Image Image Ceph Filesystem Components Abstractions Ceph rgw S3/Swift Object Store mgr Rook Operator CSI plugins osdosdganesha NFS CephCluster Block Pool Object Store Filesystem NFS Object Store User Provisioners rbd-mirror @olgch; @kublr
  • 24.
    Managed / Self-HostedOptions File system: AWS EFS; Rook; OpenEBS; GlusterFS Database: AWS RDS; MySql; PostgreSQL Ingress: AWS ELB/ALB/NLB; Ingress controller Certificates: AWS Certs; Let’s Encrypt; manual; self-signed Messaging: AWS SQS; RabbitMQ; ActiveMQ Azure has equivalent managed services for most app needs @olgch; @kublr
  • 25.
    Gotchas • Self-hosted ismore difficult to operate than managed • Different implementations have varying functionalities and QoS • Performance • Standards compliance For example • AWS EBS is AZ local • Let’s Encrypt limits certificate issuance rate • Managed services may be better hardware tuned • Self-hosted services may be better application tuned @olgch; @kublr
  • 26.
    Takeaways • Cloud nativeKubernetes applications are portable, and easy to test, experiment, and configure • Portability tools • Helm configuration templating • Kubernetes abstractions: PV, PVC, Ingress, Service etc • Using self-hosted resources where managed are not available • Managed vs self-hosted services considerations • Different platforms • On AWS and Azure • On Ubuntu and RHEL • Different ingress options • With and without reverse-proxy • With and without SSL • Different persistence options • Ephemeral storage • Managed database • Managed block storage, self-hosted database • Self-hosted cloud native storage @olgch; @kublr
  • 27.
  • 28.