The document discusses the evolution and comparison of container platforms, particularly Kubernetes and OpenShift, emphasizing their roles in DevOps and application deployment. It highlights the benefits of using containers, such as easy scalability, enhanced security, and the necessity of a cloud-native mindset, along with scenarios illustrating the advantages over traditional methods. The conclusion identifies Kubernetes as the leading container orchestrator, with OpenShift adding additional security and deployment ease.

1. Kubernetes or OpenShift - choosing
container platform for Dev and Ops
Tomasz Cholewa
DevOpsDays Warsaw 2017

2. DevOps Enthusiast
Infrastructure as Code
practitioner
Passionate learner &
certificate collector
Lead Cloud Architect
@
# whoami

3. What is container?

4. What is container?
It’s 2017 :-)

5. In case you’ve been gone for
past 3 years…

6. Not only Docker
e.g. RKT, CRI-O
Linux (mostly)
Isolation:
- Namespaces
- Cgroups

7. Container revolution
Virtual Machines Containers Serverless
VMware
Xen
KVM
LXC
Docker
RKT
AWS Lambda
Azure Functions
Google Cloud Functions

8. Container revolution
Virtual Machines Containers Serverless
VMware
Xen
KVM
LXC
Docker
RKT
AWS Lambda
Azure Functions
Google Cloud Functions
Cloud

9. CloudFoundry survey (September 8, 2017),
https://www.cloudfoundry.org/wp-content/uploads/2012/02/Container-
Report-2017-1.pdf
25% of companies are using containers in
production

10. Brave pioneers

11. Brave pioneers
Uber
Lyft
Twitter
PayPal
General
Motors
BBC News
Spotify
ebay allegro
GitHub

12. Brave pioneers
Uber
Lyft
Twitter
PayPal
General
Motors
BBC News
Spotify
ebay allegro
GitHub
And many more expected
in coming months..

13. Life without containers

14. Life without containers
Virtual
Machines
Cheap and easy
Great Isolation
Huge and reliable ecosystem

15. Life without containers
Virtual
Machines
Cheap and easy
Great Isolation
Huge and reliable ecosystem
Containers
Not everything is container
ready
Some workloads won’t
benefit
Aren’t magic pill

16. Life without containers
Virtual
Machines
Cheap and easy
Great Isolation
Huge and reliable ecosystem
Containers
Not everything is container
ready
Some workloads won’t
benefit
Aren’t magic pill
Windows
Stay where you are ;-)
Wait for better container
support

17. Containers
offer

18. Containers
offer
Single format for complete app

19. Containers
offer
Single format for complete app
Express setup of whole stack (e.g. docker-compose)

20. Containers
offer
Single format for complete app
Express setup of whole stack (e.g. docker-compose)
Innovation at speed

21. Containers
offer
Single format for complete app
Express setup of whole stack (e.g. docker-compose)
Innovation at speed
Increased security (less is more)

22. Containers
require/
encourage

23. Containers
require/
encourage
Stateless applications

24. Containers
require/
encourage
Stateless applications
Microservices

25. Containers
require/
encourage
Stateless applications
Microservices
Linux

26. Containers
require/
encourage
Stateless applications
Microservices
Linux
Cloud Native mindset

27. Containers
forbid/
discourage

28. Containers
forbid/
discourage
Tampering with running containers

29. Containers
forbid/
discourage
Tampering with running containers
Putting everything together (data, logs multiple processes)

30. Containers
forbid/
discourage
Tampering with running containers
Putting everything together (data, logs multiple processes)
Assume it will always work on the same node

31. Containers
forbid/
discourage
Tampering with running containers
Putting everything together (data, logs multiple processes)
Assume it will always work on the same node
Hardware coupling

32. Container Orchestrator
Engines

33. Container Orchestrator
Engines
1. Easy Scalability

34. Container Orchestrator
Engines
1. Easy Scalability 2. Real Portability

35. Container Orchestrator
Engines
1. Easy Scalability 2. Real Portability
3. Enhanced Security

36. Container Orchestrator
Engines
1. Easy Scalability 2. Real Portability
3. Enhanced Security 4. Forced Consistency

37. Using containers without
COE

38. Using containers without
COE
Example valid reasons

39. Using containers without
COE
Example valid reasons
You want to use as better packaging system

40. Using containers without
COE
Example valid reasons
You want to use as better packaging system
Your app doesn’t scale, is legacy or hardware constrained

41. Using containers without
COE
Example valid reasons
You want to use as better packaging system
Your app doesn’t scale, is legacy or hardware constrained
You wait for something even better or wrote your own :-)

42. Using containers without
COE
Example valid reasons
You want to use as better packaging system
Your app doesn’t scale, is legacy or hardware constrained
You wait for something even better or wrote your own :-)
Just stick with good old VMs

43. Kubernetes

44. Kubernetes
Project initiated by

45. Kubernetes
Project initiated by
Based on Borg

46. Kubernetes
Project initiated by
Based on Borg
Part of

47. Kubernetes
Project initiated by
Based on Borg
Part of
Has become the standard for COE

48. Kubernetes
Project initiated by
Based on Borg
Part of
Has become the standard for COE
Increasing adoption (Docker, GitHub)

49. OpenShift
Project
Origin

50. OpenShift
Project Products
Origin

51. OpenShift

52. OpenShift
Current release 3.6 includes Kubernetes 1.6

53. OpenShift
Current release 3.6 includes Kubernetes 1.6
Version 3 based on Kubernetes

54. OpenShift
Current release 3.6 includes Kubernetes 1.6
Version 3 based on Kubernetes
3.0 Released before Kubernetes 1.0 !

55. OpenShift
Current release 3.6 includes Kubernetes 1.6
Version 3 based on Kubernetes
3.0 Released before Kubernetes 1.0 !
Red Hat as one of the main contributors (11 of 28 leads
of K8S Special Interest Groups)

56. Kubernetes < OpenShift

57. Big Project,
General purpose
Kubernetes < OpenShift

58. Big Project,
General purpose
Project and Product
with a support
Platform for
containerized apps
Kubernetes < OpenShift

59. Kubernetes vs. OpenShift

60. Installation and operation
Easy
Multiple tools and projects
Quick run using minikube
Command line: kubectl
Optional web dashboard as add-on
Relatively easy
Manual or with Ansible
Quick run using minishift
Command line: oc
Built-in, rich web interface

63. Platforms
Supports many distros
Docker Enterprise Edition support!
Hosted versions: Google, Azure
Container runtimes (via CRI): Docker, RKT
Only RHEL, CentOS, Fedora
Hosted versions: Online or Dedicated (by
Red Hat)
Container runtimes: Docker, CRI-O (in
progress)

64. Facilities

65. Facilities
Logging: DYI
Monitoring: Heapster, cAdvisor + your own
(e.g. Prometheus)
Container registry: optional as add-on

66. Facilities
Logging: DYI
Monitoring: Heapster, cAdvisor + your own
(e.g. Prometheus)
Container registry: optional as add-on
Logging: EFK (ElasticSearch-Fluentd-
Kibana) or DYI
Monitoring: Heapster, cAdvisor + Hawkular
(JVM+Cassandra)
Container registry: integrated

67. Networking

68. Networking
SDN plugins (via CNI):
Cilium, Contiv, Contrail, Flannel, GCE,
NSX-T, Nuage, OpenVSwitch, OVN, Calico,
Romana, Weave Net
Traffic management via Ingress based on
Nginx

69. Networking
SDN plugins (via CNI):
Cilium, Contiv, Contrail, Flannel, GCE,
NSX-T, Nuage, OpenVSwitch, OVN, Calico,
Romana, Weave Net
Traffic management via Ingress based on
Nginx
SDN support (Kubernetes): Cisco Contiv,
Juniper Contrail, Flannel, VMware NSX-T,
Nokia Nuage, Tigera Calico
Custom SDN: ovs-subnet, ovs-
multitenant
Traffic management via Routes based on
HAproxy or F5

70. Security
RBAC predefined roles:

71. Security
RBAC predefined roles:

72. Security
RBAC predefined roles:
RBAC predefined roles:

73. PodSecurityPolicy

74. Security

75. Security
$ kubectl get psp --all-namespaces
No resources found.

76. Security
$ kubectl get psp --all-namespaces
No resources found.
$ oc get scc
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP
SUPGROUP PRIORITY READONLYROOTFS VOLUMES
anyuid false [] MustRunAs RunAsAny RunAsAny
RunAsAny 10 false [configMap downwardAPI emptyDir
persistentVolumeClaim secret]
hostaccess false [] MustRunAs MustRunAsRange MustRunAs
RunAsAny <none> false [configMap downwardAPI emptyDir hostPath
persistentVolumeClaim secret]
hostmount-anyuid false [] MustRunAs RunAsAny RunAsAny
RunAsAny <none> false [configMap downwardAPI emptyDir hostPath nfs
persistentVolumeClaim secret]
hostnetwork false [] MustRunAs MustRunAsRange MustRunAs
MustRunAs <none> false [configMap downwardAPI emptyDir
persistentVolumeClaim secret]
nonroot false [] MustRunAs MustRunAsNonRoot RunAsAny
RunAsAny <none> false [configMap downwardAPI emptyDir
persistentVolumeClaim secret]
privileged true [*] RunAsAny RunAsAny RunAsAny
RunAsAny <none> false [*]
restricted false [] MustRunAs MustRunAsRange MustRunAs
RunAsAny <none> false [configMap downwardAPI emptyDir

77. Security
Default policy: OPEN Default policy forbids container to:
Run as root user
Run as privileged container
Access volumes from host
Set additional CAPS

78. App deployment

79. App deployment
Deployment unit: Pod
Helm with charms

80. App deployment
Deployment unit: Pod
Helm with charms
Deployment unit: Pod
Templates

81. App deployment

82. App deployment
ReplicateSet, ReplicaController
Deployment
Strategy: Recreate, RollingUpdate
Canary, BlueGreen
ReplicateSet, ReplicaController
Deployment, DeploymentConfig
Strategy: Recreate, RollingUpdate,
Custom
Canary, BlueGreen, A/B Testing
Dedicated object for building: Build,
BuildConfig

83. Dev and Ops collaboration

84. 2007

85. 2017
DEV

86. Apps

87. Apps
Business

88. Apps
Business
Provide apps that
meet business
requirements
Dev

89. Apps
Business
Provide apps that
meet business
requirements
Dev
Provide
environment for
those apps
Ops

90. Apps
Business
Provide apps that
meet business
requirements
Dev
Provide
environment for
those apps
Ops
Make applications
available to users in most
reliable, secure and
efficient way

91. Scenarios

92. 1. Application is slow

93. Legacy
approach
ssh 10.12.13.14 -l root
tail -f /var/log/syslog
df
du
killall -9 java
reboot

94. Legacy
approach
ssh 10.12.13.14 -l root
tail -f /var/log/syslog
df
du
killall -9 java
reboot
Dirty tricks
Make it work and forget
Now it’s up to Dev to fix it

95. Container
approach

96. Container
approach
Use container and orchestrator metrics

97. Container
approach
Use container and orchestrator metrics
Applications metrics

98. Container
approach
Use container and orchestrator metrics
Applications metrics
Easily Scale and autoscale

99. Container
approach
Use container and orchestrator metrics
Applications metrics
Easily Scale and autoscale
Fix once and deploy everywhere

100. Container
approach Pod
Pod
Pod
Pod
Metrics
HorizontalPodAutoscaler
Fetch
Scale
Autoscaling

101. Container
approach Pod
Pod
Pod
Pod
Metrics
HorizontalPodAutoscaler
Heapster+cAdvisor
Prometheus
Hawkular (OpenShift)
Fetch
Scale
Autoscaling

102. Container
approach Pod
Pod
Pod
Pod
Metrics
HorizontalPodAutoscaler
Heapster+cAdvisor
Prometheus
Hawkular (OpenShift)
CPU
Custom metrics (beta)
Fetch
Scale
Autoscaling

103. 2. Application has crashed

104. 2. Application has crashed
again

105. Legacy
approach
ssh 10.12.13.14 -l root
tail -f /var/log/syslog
df
du
killall -9 java
reboot

106. Legacy
approach
It’s java
It’s Dev fault
ssh 10.12.13.14 -l root
tail -f /var/log/syslog
df
du
killall -9 java
reboot

107. Container
approach

108. Container
approach
Do nothing - let orchestrator take care of it (livenessProbe)

109. Container
approach
Do nothing - let orchestrator take care of it (livenessProbe)
Use central logging

110. Container
approach
Do nothing - let orchestrator take care of it (livenessProbe)
Use central logging
Rollback to previous version

111. Container
approach
Do nothing - let orchestrator take care of it (livenessProbe)
Use central logging
Rollback to previous version
Fix once and deploy

112. 3. Mesh of interconnected
applications

113. Legacy
approach
App1
App2 App3
App4
App5

114. Legacy
approach
App1
App2 App3
App4
App5
IP + /etc/hosts
Maybe load balancer
DHCP
Manually configured DNS

115. Container
approach
Pod Pod Pod Pod Pod
Service
Ingress
Cloud Load
Balancers

116. Container
approach
Pod Pod Pod Pod Pod
Service
Ingress
Cloud Load
Balancers
DNS (`*.cluster.local)
Group by label

117. Container
approach
Pod Pod Pod Pod Pod
Service
Ingress
Cloud Load
Balancers
DNS (`*.cluster.local)
Group by label
Service
Pod

118. Container
approach
Pod Pod Pod Pod Pod
Service
Ingress Router
Cloud Load
Balancers
DNS (`*.cluster.local)
Group by label
HAproxy
A/B Testing
Service
Pod

119. Container
approach
Pod Pod Pod Pod Pod
Service
F5-BIGIPIngress Router
Cloud Load
Balancers
DNS (`*.cluster.local)
Group by label
HAproxy
A/B Testing
Native integration (via
API >=12.*)
Service
Pod

120. 3. Need to store data from my
apps

121. Legacy
approach “It will be ready next week, need to configure access”
Your favourite and busy Ops

122. Legacy
approach “It will be ready next week, need to configure access”
Your favourite and busy Ops
Why so long?

123. Legacy
approach “It will be ready next week, need to configure access”
Your favourite and busy Ops
Why so long? Why so slow?

124. Legacy
approach “It will be ready next week, need to configure access”
Your favourite and busy Ops
Why so long? Why so slow?
“I’ll just save it in /opt/myapp/lib/2.1/
inconspicious_dir/critical_data”

125. Legacy
approach “It will be ready next week, need to configure access”
Your favourite and busy Ops
I’m pretty sure it’s
being backed up!
Why so long? Why so slow?
“I’ll just save it in /opt/myapp/lib/2.1/
inconspicious_dir/critical_data”

126. Container
approach
PersistentVolumeClaim
PersistentVolume
Pod
Request
Match & allocate
Use

127. Container
approach
PersistentVolumeClaim
NFS
iSCSI
Ceph
Glustterfs
Cinder
AWS EBS
GCE PD
vSpher VMDK
Azure Disk
…
PersistentVolume
Plugins
Pod
Request
Match & allocate
Use

128. Container
approach
PersistentVolumeClaim
NFS
iSCSI
Ceph
Glustterfs
Cinder
AWS EBS
GCE PD
vSpher VMDK
Azure Disk
…
PersistentVolume
Plugins
Pod
Request
Match & allocate
Use
superfast!

129. Container
approach
PersistentVolumeClaim
NFS
iSCSI
Ceph
Glustterfs
Cinder
AWS EBS
GCE PD
vSpher VMDK
Azure Disk
…
PersistentVolume
Plugins
Pod
Request
Match & allocate
Use
superfast!
superslow,
but huge

130. Container
approach
PersistentVolumeClaim
NFS
iSCSI
Ceph
Glustterfs
Cinder
AWS EBS
GCE PD
vSpher VMDK
Azure Disk
…
PersistentVolume
Plugins
Pod
Request
Match & allocate
Use
superfast!
superslow,
but huge
Dynamic
and
Static

131. 4. Another heartbleed-type
bug has appeared

132.
Legacy
approach

133.
Legacy
approach
ssh 10.12.13.14 -l root
apt-get update && apt-get -y update &&
reboot

134. DeploymentConfig
Container
approach
ImageStream
New image
Dev
Test
Prod
Trigger
Redeploy

135. DeploymentConfig
Container
approach
ImageStream
New image
Dev
Test
Prod
Trigger
Redeploy
Deplopyment
hooks:
pre, mid, post

136. 5. Application provided by
external vendor

137. Legacy
approach
scp awesome-app-SNAPSHOT.jar srv:~/app/
ansible -i all web -m service
-a “name=tomcat state=restarted”

138. Legacy
approach
Unknown quality with murky cumbersome instruction
Environments like snowflakes
Not this time - wait for new one (“a couple of days”)
scp awesome-app-SNAPSHOT.jar srv:~/app/
ansible -i all web -m service
-a “name=tomcat state=restarted”

139. Container
approach
oc new-app myrepo/myapp:1.0
MyProject
Create
Pod
Service
Route

140. Container
approach
oc new-app myrepo/myapp:1.0
MyProject
Create
Pod
Service
Route
Create directly from:
Docker Image
Source Code
Template

141. Container
approach
oc new-app myrepo/myapp:1.0
MyProject
Create
Pod
Service
Route
Create directly from:
Docker Image
Source Code
Template
Also builds image
automatically from source
with autodetection of
used language!

142. Container
approach
oc new-app myrepo/myapp:1.0
MyProject
Create
Pod
Service
Route
Create directly from:
Docker Image
Source Code
Template
Also builds image
automatically from source
with autodetection of
used language!
Create on
demand

143. 6. Is my new app secure
enough for public release?

144. Legacy
approach
Of course it is!
Dev
Hell no!
Ops

145. Legacy
approach
Of course it is!
Dev
Hell no!
Ops
Ship it!
Business

146. Container
approach
Continuous Deployment Pipeline

147. Container
approach
Continuous Deployment Pipeline
Automatic with ManageIQ/
CloudForms

148. Container
approach
Build
&
publish
Test Deploy
Dev
ImageStreamImagePolicy
Internal
Registry
Docker
Hub
External
Registry
Deploy
Test
Deploy
Prod
Dev
Test
Prod
CD Pipeline

149. Container
approach

150. Container
approach
Jenkins inside
with on-demand
slaves

151. Quick summary

152. 1
You can live without
containers just fine

153. 2
You can use containers
without orchestrators, but
it’s pointless

154. 3
Kubernetes has won the
war and it’s the best and
most versatile container
orchestrator

155. 4
OpenShift = Kubernetes
+ security rules + better
deployment

156. 5
Cloud, container and
serverless require Cloud
Native mindset

157. Thank you!
Contact
tomasz@cloudowski.com