Kubernetes is an amazing technology, but getting it up and running in your data center or VMs is challenging. In this technical webinar, you will learn how best to deploy, operate, and scale Kubernetes clusters from one to hundreds of nodes using DC/OS. Jörg Schad and Adrian Smolski from Mesosphere show how to run Kubernetes on DC/OS, as well as how to integrate and run Kubernetes alongside traditional applications and fast data services of your choice (e.g. Apache Cassandra, Apache Kafka, Apache Spark, TensorFlow, and more) on any infrastructure. You will learn how to: 1. Deploy Kubernetes in a secure, highly available, and fault-tolerant manner on DC/OS 2. Solve operational challenges of running a large/multiple Kubernetes cluster(s) 3. One-click deploy big data stateful and stateless services alongside a Kubernetes cluster Jörg is a Technical Lead for Community Projects at Mesosphere in San Francisco. His speaking experience includes various Meetups, international conferences, and lecture halls. Adrian Smolski is the local Field CTO based out of Sydney, Australia. His background is big data, data science and distributed systems.

1. Operating Kubernetes at Scale
@joerg_schad @dcos

2. Jörg Schad
Technical Community
Lead / Developer
● Core Mesos
developer at
Mesosphere
● Passions are deep
learning, distributed
data systems, and
data analytics

3. Adrian Smolski
Field CTO, ANZ
● Previously mainly
focussed on Big Data
& Data Science
● Worked in-house, in
consulting and now in
the vendor world

4. © 2017 Mesosphere, Inc. All Rights Reserved. 4
In the beginning
there was a big
Monolith

5. © 2016 Mesosphere, Inc. All Rights Reserved. 5

6. © 2018 Mesosphere, Inc. All Rights Reserved.
Hardware
Operating System
Application
6
COMPUTERS

7. © 2018 Mesosphere, Inc. All Rights Reserved.
noun | ˈmīkrō/ /ˈsərvəs/ :
an approach to application development in which a
large application is built as a suite of modular services.
Each module supports a specific business goal and
uses a simple, well-defined interface to
communicate with other modules.*
Microservices are designed to be flexible, resilient,
efficient, robust, and individually scalable.
*From whatis.com
OVERVIEW

8. © 2018 Mesosphere, Inc. All Rights Reserved.
Operating
System
Operating
System
Operating
System
ServiceApp ServiceServiceAppApp
8
MICROSERVICE
S
- Polyglot
- Single
Responsibility
- Smaller Teams
- Utilization
- Machine
types/groups
- Dependency hell
Machine
Infrastructure
Machine Machine
ServiceService ServiceServiceServiceService

9. © 2017 Mesosphere, Inc. All Rights Reserved. 9
Container …
!=
• container runtime* != container image != container instance

10. © 2018 Mesosphere, Inc. All Rights Reserved.
ServiceApp ServiceServiceAppApp
OS
10
CONTAINERS
- Rapid deployment
- Dependency
vendoring
- Container image
repositories
- Spreadsheet
scheduling
OS OS
Machine
Infrastructure
Machine Machine
Container Runtime Container Runtime Container Runtime
ServiceService ServiceServiceServiceService

11. © 2018 Mesosphere, Inc. All Rights Reserved. 11
CONTAINER
SCHEDULING
RESOURCE
MANAGEMENT
SERVICE
MANAGEMENT
- Load Balancing
- Readiness Checking
CONTAINER ORCHESTRATION

12. © 2018 Mesosphere, Inc. All Rights Reserved. 12
CONTAINER
SCHEDULING
- Placement
- Replication/Scaling
- Resurrection
- Rescheduling
- Rolling Deployment
- Upgrades
- Downgrades
- Collocation
RESOURCE
MANAGEMENT
- Memory
- CPU
- GPU
- Volumes
- Ports
- IPs
- Images/Artifacts
SERVICE
MANAGEMENT
- Labels
- Groups/Namespaces
- Dependencies
- Load Balancing
- Readiness Checking
CONTAINER ORCHESTRATION

13. © 2018 Mesosphere, Inc. All Rights Reserved.
Orchestration
13
Machine Infrastructure
Web Apps & Services
Scheduling
Resource Management
Container Runtime
Machine & OS
Service Management
CONTAINER
ORCHESTRATIO
N
Machine & OS Machine & OS
Container Runtime Container Runtime

14. © 2017 Mesosphere, Inc. All Rights Reserved. 14
Kubernetes
Overview

15. Container
Pods

16. Volume
Data
Loader
Front
End
Pods

17. Volume
Data
Loader
Front
End
Pods
Set of Related Containers
Tightly Coupled
Ephemeral/Stateless
Share Namespace
Share Network
Unit of Management

18. Pods
Set of Related Containers
Tightly Coupled
Ephemeral/Stateless
Share Namespace
Share Network
Unit of Management
Pod

19. Pod Networking
D
B
C
A
Routable
(layer 3)
No NAT
(internode)10.0.0.1 10.0.0.2
10.0.0.3 10.0.0.4

20. Labels
D
B
C
A
app = portal
tier = frontend
version = v1
app = portal
tier = backend
version = v1
app = portal
tier = frontend
version = v2
app = portal
tier = backend
version = v2

21. Label Selectors
D
B
C
A
app = portal
tier = frontend
version = v1
app = portal
tier = backend
version = v1
app = portal
tier = frontend
version = v2
app = portal
tier = backend
version = v2
app = portal
SELECTOR

22. © 2018 Mesosphere, Inc. All Rights Reserved.
KUBERNETES Concepts
kind: Deployment
kind: ReplicaSet
kind: Pod
scales
N instances
of Pod
Set of
Containers
Upgrades
deployes
22

23. © 2018 Mesosphere, Inc. All Rights Reserved.
Configuration File
apiVersion: apps/v1beta2
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
23

24. © 2018 Mesosphere, Inc. All Rights Reserved.
Configuration File
apiVersion: apps/v1beta2
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
ReplicaSet
Pod
24

25. ETCD 01
MASTER 01
etcd
LoadBalancer
WORKER 01
API Server
Scheduler
Controller
Manager
Kubectl
Kubelet Kube Proxy
Container Engine (Docker, CRI-
O)
Pod Pod Pod
HTTPS
HTTPS
HTTPS
HTTPS

26. © 2017 Mesosphere, Inc. All Rights Reserved. 26
Challenges

27. © 2018 Mesosphere, Inc. All Rights Reserved. 27
Kubernetes ✕ 2
Machine Infrastructure
Container Runtime
Machine & OS Machine & OS Machine & OS
Container Runtime
Container
Runtime
Container Runtime
Machine & OS Machine & OS
Container Runtime
Orchestration
Scheduling
Resource Management
Service Management
Web Apps & Services
Machine Infrastructure
OrchestrationScheduling
Resource Management
Service Management
Web Apps & Services

28. © 2018 Mesosphere, Inc. All Rights Reserved. 28
Deployments
Machine Infrastructure
Container Runtime
Machine & OS Machine & OS Machine & OS
Container Runtime
Container
Runtime
Container Runtime
Machine & OS Machine & OS
Container Runtime
Machine Infrastructure
Web Apps & Services
etcd etcdetcd
api-
server
api-
server
api-
server
Kube-
proxy
Kube-
proxy
scheduler
Kube-
proxy
scheduler
schedulercontrollercontroller
controller
kubelet kubelet kubelet
...

29. © 2018 Mesosphere, Inc. All Rights Reserved. 29
Failures
Machine Infrastructure
Container Runtime
Machine & OS Machine & OS Machine & OS
Container Runtime
Container
Runtime
Container Runtime
Machine & OS Machine & OS
Container Runtime
Machine Infrastructure
Web Apps & Services
etcd etcdetcd
api-
server
api-
server
api-
server
Kube-
proxy
Kube-
proxy
scheduler
Kube-
proxy
scheduler
schedulercontrollercontroller
controller
kubelet kubelet kubelet
...

30. 3 AM
Typical Datacenter
siloed, over-provisioned servers,
low utilization
Kubernetes
Jenkins
Kafka
Spark
Cassandra

31. © 2017 Mesosphere, Inc. All Rights Reserved. 31
Datacenter

32. © 2017 Mesosphere, Inc. All Rights Reserved. 32
Datacenter
Typical Datacenter
siloed, over-provisioned servers,
low utilization
Kubernetes
Jenkins
Kafka
Spark
CockroachDB

33. © 2017 Mesosphere, Inc. All Rights Reserved. 33

34. © 2017 Mesosphere, Inc. All Rights Reserved. 34
Datacenter
Typical Datacenter
siloed, over-provisioned servers,
low utilization
Mesos/ DC/OS
automated schedulers, workload multiplexing onto the
same machines
Tensorflow
Jenkins
Kafka
Spark
Tensorflow

35. PHYSICAL INFRASTRUCTURE
MICROSERVICES, CONTAINERS, & DEV TOOLS
VIRTUAL MACHINES PUBLIC CLOUDS
DATA SERVICES, MACHINE LEARNING, & AI
Security &
Compliance
Application-Aware
Automation
Multitenancy
Hybrid Cloud
Management
100+
MORE
DatacenterEdge
Datacenter and Cloud as a Single Computing Resource
Powered by Apache Mesos
20+
MORE
Unified hybrid cloud operations
Securely manage cloud, datacenter, and edge
infrastructures from a single control plane
4
Mesosphere DC/OS
Intelligent resource pooling
Optimize workload density for highest utilization with
resource guarantees
3
Broad workload coverage
Run today & tomorrow’s applications including traditional
J2EE, containers, analytics & ML
1
Application-aware automation
Automate workload-specific operating procedures to “as-a-
Service” anything from Kubernetes to data services
2

36. © 2018 Mesosphere, Inc. All Rights Reserved.
Two-level Scheduling
1. Agents advertise resources to Master
2. Master offers resources to Framework
3. Framework rejects / uses resources
4. Agent reports task status to Master
36
MESOS ARCHITECTURE
Mesos
Master
Mesos
Master
Mesos
Master
Mesos AgentMesos Agent Service
Cassandra
Executor
Cassandra
Task
Kubernetes
Scheduler
Spark
Executor
Spark
Task
Mesos AgentMesos Agent Service
Docker
Executor
Docker
Task
K8s Executor
Kubelet
Task
Marathon
Scheduler
Kafka
Scheduler

37. © 2018 Mesosphere, Inc. All Rights Reserved. 37
Mesos and Kubernetes
Machine Infrastructure
Container Runtime
Machine & OS Machine & OS Machine & OS
Container Runtime
Container
Runtime
Container Runtime
Machine & OS Machine & OS
Container Runtime
Orchestration
Scheduling
Resource Management
Service Management
Web Apps & Services
Machine Infrastructure
DC/OS / Apache Mesos
OrchestrationScheduling
Resource Management
Service Management
Web Apps & Services

38. © 2018 Mesosphere, Inc. All Rights Reserved. 38
Mesos and Kubernetes
Orchestration
Machine Infrastructure
Web Apps & Services
Scheduling
Resource Management
Container Runtime
Machine & OS
Service Management
Machine & OS Machine & OS
Container Runtime
Container
Runtime
Container Runtime
Machine & OS Machine & OS
Container Runtime
Orchestration
Scheduling
Resource Management
Service Management
Web Apps & Services
Machine Infrastructure
DC/OS / Apache Mesos
Big Data
Services
● Spark
● Flink
● Kafka
● ….

39. © 2018 Mesosphere, Inc. All Rights Reserved. 39
Why {Spark, HDFS, ..} on K8s today?
Kelsey Hightower
Kubernetes Thought
Leader
Ranked #1 K8s Influencer
Staff Developer
Advocate
PM & Chief Advocate
Today Big Data on K8s is more DIY
Top
Kubernetes
Advocate

40. © 2018 Mesosphere, Inc. All Rights Reserved. 40
Mesos and Kubernetes
Machine Infrastructure
Container Runtime
Machine & OS Machine & OS Machine & OS
Container Runtime
Container
Runtime
Container Runtime
Machine & OS Machine & OS
Container Runtime
Machine Infrastructure
DC/OS / Apache Mesos
Web Apps & Services
etcd etcdetcd
api-
server
api-
server
api-
server
Kube-
proxy
Kube-
proxy
scheduler
Kube-
proxy
scheduler
schedulercontrollercontroller
controller
kubelet kubelet kubelet
...

41. © 2018 Mesosphere, Inc. All Rights Reserved. 41
Mesos and Kubernetes
Machine Infrastructure
Container Runtime
Machine & OS Machine & OS Machine & OS
Container Runtime
Container
Runtime
Container Runtime
Machine & OS Machine & OS
Container Runtime
Machine Infrastructure
DC/OS / Apache Mesos
Web Apps & Services
etcd etcdetcd
api-
server
api-
server
api-
server
Kube-
proxy
Kube-
proxy
scheduler
Kube-
proxy
scheduler
schedulercontrollercontroller
controller
kubelet kubelet kubelet
...

42. © 2017 Mesosphere, Inc. All Rights Reserved. 42
DC/OS
Kubernetes
Integration

43. © 2018 Mesosphere, Inc. All Rights Reserved.
● Unaltered upstream Kubernetes, unlike OpenShift
● Stand-alone Kubernetes clusters are exposed, not a PaaS using Kubernetes
● Simple install onto a DC/OS cluster
● Integration with DC/OS data services
● HA and Secure by default in EE
● Customers have choice between Marathon and Kubernetes
DC/OS Kubernetes
43

44. © 2018 Mesosphere, Inc. All Rights Reserved. 44
Networking in Kubernetes on DC/OS

45. © 2018 Mesosphere, Inc. All Rights Reserved. 45
Demo
Generator Display
1. Financial data created
by generator
2. Written to
Kafka topics
3. Kafka Topics
consumed by Spark or
Flink
4. Results written back into Kafka
stream (another topic)
7. Results displayed

46. © 2018 Mesosphere, Inc. All Rights Reserved. 46
Demo
Generator Display
1. Financial data created
by generator
2. Written to
Kafka topics
4. Results written back into Kafka
stream (another topic)
5. Results displayed
3. Kafka Topics
consumed and analyzed
by Flink
Kubernetes Cluster
(running on top of DC/OS)

47. Download Now
https://mesosphere.com/resource/category/ebook/

48. © 2018 Mesosphere, Inc. All Rights Reserved. 48
THANK YOU!
ANY
QUESTIONS?
@dcos
users@dcos.io
/groups/8295652
/dcos
/dcos/examples
/dcos/demos
chat.dcos.io
https://github.com/mesosphere/dcos-kubernetes-quickstart
https://mesosphere.com/blog/another-kubernetes-service/