There is a transformation brewing for DevOps in age of Kubernetes. The tools of the trade, configuration management solutions, have been superseded in agility and preference by development teams who want the declarative choreography of containerized applications. The new preference for mixing developer and operations is the site reliability engineering (SRE) model championed by Google. In this new structure, the need to automate doesn’t stop at the containerized application and DevOps professionals should seek to automate the Kubernetes service itself.

1. DevOps in Age of Kubernetes

2. Chris Gaun
Kubernetes Product @
Mesosphere /
Kubernetes Expert /
CNCF Ambasador
● Previous to that
Gartner analyst
covering public IaaS
● Kubernetes
community for 3 years

3. 3
Sign Up For Slack
1. Slack URL: https://chat.dcos.io/
2. Join #kubernetes channel
3. OSS support / feedback
4. Search “DC/OS Kubernetes
Quickstart Github” or
https://github.com/mesosphere/d
cos-kubernetes-quickstart

4. 4
Star / Clone Github
1. Go to Kubernetes DC/OS
quickstart
2. Search “DC/OS Kubernetes
Quickstart Github” or
https://github.com/mesospher
e/dcos-kubernetes-quickstart
3. Live demo
https://github.com/dcos/demos
/tree/master/flink-k8s/1.11

5. Engine crank to start Turn key
Shift into gear
First Cars Cars Today
Automatic
transmission
Drive from A to B
Self-driving…
ish
Declarative
Intelligent Resource Pooling
Evolution of Management Automation

6. © 2017 Mesosphere, Inc. All Rights Reserved. 6
In the beginning there was ….

7. © 2018 Mesosphere, Inc. All Rights Reserved.
Hardware
Operating System
Application
7
Monolithic
model

8. © 2017 Mesosphere, Inc. All Rights Reserved. 8
Things get smaller

9. © 2017 Mesosphere, Inc. All Rights Reserved. 9
Connect some smaller computers to the big one ...

10. © 2017 Mesosphere, Inc. All Rights Reserved. 10
Scaling ...

11. © 2017 Mesosphere, Inc. All Rights Reserved. 11
MapReduce ...

12. © 2017 Mesosphere, Inc. All Rights Reserved. 12
Static partitioning

13. © 2017 Mesosphere, Inc. All Rights Reserved. 13
Static partitioning

14. © 2017 Mesosphere, Inc. All Rights Reserved. 14
Virtualisation
Hardware
Operating System
App
Hypervisor
App
Operating System
App App
Hardware
Operating System
App
Hypervisor
App
Operating System
App App

15. © 2018 Mesosphere, Inc. All Rights Reserved.
noun | ˈmīkrō/ /ˈsərvəs/ :
an approach to application development in which a
large application is built as a suite of modular services.
Each module supports a specific business goal and
uses a simple, well-defined interface to
communicate with other modules.*
Microservices are designed to be flexible, resilient,
efficient, robust, and individually scalable.
*From whatis.com
Microservices

16. © 2018 Mesosphere, Inc. All Rights Reserved.
Operating
System
Operating
System
Operating
System
ServiceApp ServiceServiceAppApp
16
Microservices
- Polyglot
- Single
Responsibility
- Smaller Teams
- Utilization
- Machine
types/groups
- Dependency hell
Machine
Infrastructure
Machine Machine
ServiceService ServiceServiceServiceService

17. © 2017 Mesosphere, Inc. All Rights Reserved. 17
Containers
Hardware
Operating System
App
Container Runtime
App App
Hardware
Operating System
App
Container Runtime
App App

18. © 2018 Mesosphere, Inc. All Rights Reserved.
ServiceApp ServiceServiceAppApp
OS
18
Containers
- Rapid deployment
- Dependency
vendoring
- Container image
repositories
- Spreadsheet
scheduling
OS OS
Machine
Infrastructure
Machine Machine
Container Runtime Container Runtime Container Runtime
ServiceService ServiceServiceServiceService

19. © 2018 Mesosphere, Inc. All Rights Reserved. 19
CONTAINER
SCHEDULING
RESOURCE
MANAGEMENT
SERVICE
MANAGEMENT
- Load Balancing
- Readiness Checking
Container Orchestration

20. © 2018 Mesosphere, Inc. All Rights Reserved. 20
CONTAINER
SCHEDULING
- Placement
- Replication/Scaling
- Resurrection
- Rescheduling
- Rolling Deployment
- Upgrades
- Downgrades
- Collocation
RESOURCE
MANAGEMENT
- Memory
- CPU
- GPU
- Volumes
- Ports
- IPs
- Images/Artifacts
SERVICE
MANAGEMENT
- Labels
- Groups/Namespaces
- Dependencies
- Load Balancing
- Readiness Checking
CONTAINER ORCHESTRATION

21. © 2018 Mesosphere, Inc. All Rights Reserved.
Orchestration
21
Machine Infrastructure
Web Apps & Services
Scheduling
Resource Management
Container Runtime
Machine & OS
Service Management
CONTAINER
ORCHESTRA
TION
Machine & OS Machine & OS
Container Runtime Container Runtime

23. • 2 Billion containers started
per week
• 15 + years of container
orchestration
• Learnt many, many lessons
• Distributed systems DNA
– cgroups
– artificial intelligence
– mapreduce
– bigtable

24. © 2017 Mesosphere, Inc. All Rights Reserved. 24
MapReduce is
crunching Data
Ancient
Times...

25. © 2016 Mesosphere, Inc. All Rights Reserved. 25
But then business
demanded
FAST DATA
We need to turn faster!
Today...

26. © 2016 Mesosphere, Inc. All Rights Reserved. 26
Fast Data
Batch Event ProcessingMicro-Batch
Days Hours Minutes Seconds Microseconds
Solves problems using predictive and prescriptive analyticsReports what has happened using descriptive analytics
Predictive User InterfaceReal-time Pricing and Routing Real-time AdvertisingBilling, Chargeback Product recommendations

27. © 2016 Mesosphere, Inc. All Rights Reserved. 27
The SMACK Stack
EVENTS
Ubiquitous data streams
from connected devices
INGEST
Apache Kafka
STORE
Apache Spark
ANALYZE
Apache Cassandra
ACT
Akka
Ingest millions of events
per second
Distributed & highly
scalable database
Real-time and batch
process data
Visualize data and build
data driven applications
Apache Mesos/ DC/OS
Sensors
Devices
Clients

28. © 2016 Mesosphere, Inc. All Rights Reserved. 28
The SMACK Stack
EVENTS
Ubiquitous data streams
from connected devices
INGEST
Apache Kafka
STORE
Apache Flink
ANALYZE
Apache Cassandra
ACT
Akka
Ingest millions of events
per second
Distributed & highly
scalable database
Real-time and batch
process data
Visualize data and build
data driven applications
Apache Mesos/ DC/OS
Sensors
Devices
Clients

29. © 2017 Mesosphere, Inc. All Rights Reserved. 29
Challenges

30. © 2018 Mesosphere, Inc. All Rights Reserved. 30
Datacenter
Typical Datacenter
siloed, over-provisioned servers,
low utilization
Kubernetes
Jenkins
Kafka
Spark
Cassandra

31. 3 AM
Typical Datacenter
siloed, over-provisioned servers,
low utilization
Kubernetes
Jenkins
Kafka
Spark
Cassandra

32. DC1
DC2
DC3
3 AM

33. © 2017 Mesosphere, Inc. All Rights Reserved. 33
Apache Mesos:
The datacenter
kernel
http://mesos.apache.or
g/

34. © 2017 Mesosphere, Inc. All Rights Reserved.
• A cluster resource negotiator
• A top-level Apache project
• Scalable to 10,000s of
nodes
• Fault-tolerant, battle-tested
• An SDK for distributed apps
• Native Docker support
34
Building block of the modern internet
http://mesos.apache.org/documentation/latest/powered-by-
mesos/

35. © 2017 Mesosphere, Inc. All Rights Reserved. 35
MULTIPLEXING OF DATA, SERVICES, USERS,
ENVIRONMENTS
Typical Datacenter
siloed, over-provisioned servers,
low utilization
Apache Mesos
automated schedulers, workload multiplexing onto the
same machines
mySQL
microservice
Cassandra
Spark/Hadoo
p
Kafka

36. Mesosphere Helps Businesses Navigate
Digital Transformation and Hybrid Cloud
Capture value from real-time data
Streaming data analytics, machine learning/AI to
power personalization, anomaly detection, & IoT
Ship more apps quickly
Accelerate time-to-value of new projects
Modernize enterprise architecture
Save money with automation and unified hybrid cloud
operations
● 30% of Fortune 50 companies
● 5 of top 10 banks in N. America
● 5 of top 10 telcos WW
● 5 of 10 most valued startups WW
● 6 top automotive tech companies WW

37. New Levels of Complexity
Apache Kafka message queue Kubernetes container orchestrator
Store AnalyzeTransport Serve
Apache Spark analytics engineApache Cassandra distributed DB
Infrastructure
37

38. PHYSICAL INFRASTRUCTURE
MICROSERVICES, CONTAINERS, & DEV TOOLS
VIRTUAL MACHINES PUBLIC CLOUDS
DATA SERVICES, MACHINE LEARNING, & AI
Security &
Compliance
Application-Aware
Automation
Multitenancy
Hybrid Cloud
Management
100+
MORE
DatacenterEdge
Datacenter and Cloud as a Single Computing Resource
Powered by Apache Mesos
20+
MORE
Unified hybrid cloud operations
Securely manage cloud, datacenter, and edge
infrastructures from a single control plane
4
Mesosphere DC/OS
Intelligent resource pooling
Optimize workload density for highest utilization with
resource guarantees
3
Broad workload coverage
Run today & tomorrow’s applications including traditional
J2EE, containers, analytics & ML
1
Application-aware automation
Automate workload-specific operating procedures to “as-a-
Service” anything from Kubernetes to data services
2

39. DC/OS is Run Anywhere Kubernetes Cloud Provider
Kubernetes-as-a-Service
for Each Team
Consistent Kubernetes-as-a-Service
experience for different teams, with
different versions across teams or phases
of dev/test/prod
Mesos Automates
Kubernetes
Complete lifecycle automation - Deploy
and automate the ongoing management of
the Kubernetes cluster including
installation, high availability, security,
scaling, & upgrade
Entire CI/CD Pipeline on
Shared Resources
Share resources with Jenkins and other
services in the development pipeline
ROADMAP: bin pack Kubernetes

40. DC/OS
Layers of a Kubernetes Solution
Google designed
Kubernetes for use as a
cloud service offering
Mesosphere automates
Kubernetes for use on
any infrastructure
Infrastructure:
Public cloud, virtualization, bare metal
as-a-Service Automation:
Provisioning, scaling, self-healing, upgrades
Kubernetes Service:
Containerized application orchestration
Continuous Delivery / Continuous Integration:
Host applications on Kubernetes
Development:
Agile microservices dev with container & data services
Hybrid Cloud
(Any Infra)
GKE DIY
Hybrid Cloud
(Any Infra)
Manual
operations
PaaS
Datacenter
Per PaaS
Per PaaS

41. Intelligent Classical Cloud Specific
Mesos’ Position in Container Orchestration World
● Mesos - Intelligent
resource management
● How it Automates -
Orchestrates Kubernetes
infrastructure like
Kubernetes orchestrates
applications
● Example - "(Netflix)
launches up to 500,000
containers and 200,000
clusters/day"
-
Kubernetes Titus
(Netflix)Resource
Management
Container
Management
Container
Application
Mesos’ control plane dynamically configures
infrastructure based on user’s desired states
Optimized for configuration management where
infrastructure is hard coded to environment
YouTube,
Twitter

42. How to Manage Kubernetes
Manual
- Exceptionally
difficult
- Multiple steps
- Zero day-two
management
DIY Configuration
Management
- Easy “hello world”
- No commercial
support
- DIY security
- Lacks auxiliary
components
Vendor Configuration
Management
- Easy provisioning
of clusters
- Hard coded
infrastructure
- Includes auxiliary
components
Cloud Provider
- Easy provisioning
multi-clusters
- Standard centralized
management
- Includes auxiliary
components
KOPS
Kubespray
Kubeadm
Public Anywhere
Passive Intelligent
Google Container
Engine (GKE)

43. Enabling Developer Agility:
Kubernetes is one part of a holistic CI/CD pipeline
Source Code
Control
Build and Test Release
Deploy, Monitor
and Log
Automatically trigger CI/CD
pipeline based on code check-
in.
Start automated build and test,
including functional, security
and performance tests.
Update artifact repository with
latest successful code artifacts
and pull newest images
Deploy applications to container
orchestration and watch with
monitoring and logging
CONTINUOUS INTEGRATION (CI)
CONTINUOUS DEPLOYMENT
(CD)
Marathon

44. Production Environment
● Shared but namespaced
● Trusted code
● Low compliance / lower security
● Developers have no / little
access
Development
● Individual clusters
● Full API access
● Complete debugging on
distributed system
44
USE CASE: Share Some Things (e.g. Production)
App1 App2 App3 App4
CI/CD
Dev Team 1
Dev Team 2

45. Production Environment
● Hard seperation
● Secure environments
● Microservices
● SRE best practice
Development
● Individual clusters
● Full API access
● Complete debugging on
distributed system
45
USE CASE: Share Nothing
App1 App2 App3 App4
CI/CD
Dev Team 1
Dev Team 2

46. 46
Why Multiple Clusters?
● Security and compliance
● Worried about noisy neighbor
● Worried about NOSY neighbor
● Require full API access
(development cluster)
● Multiple independent workloads
(like CERN)
● Easier to managing scaling,
provisioning, and independent
lifecycles (think cluster instead of
cluster/environment)

47. 47
USE CASE: Share Everything

48. Run Everywhere Cloud Provider For Kubernetes
Solution
Datacenter and Cloud as a Single Resource
Powered by Apache Mesos
Centralized
Operations
Networking &
Ingress
Identity
Management
Storage CSI
Provider
Line of Business 1 Line of Business 2 Developer Cluster Shared Production
Skaffold

49. Bin Packing
Multi-Kubernetes Roadmap
High Density Multiple Kubernetes
82% CPU Utilization
76% Memory Utilization
82% CPU Utilization
76% Memory Utilization
● Deliver entire Kubernetes
pipeline as-a-Service
● Use shared resources to
maximize utilization
● Roadmap: Provide each
developer group or LOB
with fully isolated and
secure Kubernetes cluster
Node 1
CPU
Node 1
CPU

50. What’s Kubernetes Done Right?
● Automated - Kubernetes operations should be automated
● Evergreen - You want to always run the latest version of Kubernetes
● Included - Kubernetes is a commoditized standard that is included in a solution
● Open source - Market standard is pure OSS Kubernetes
● Unified - Kubernetes is not an island, you’ll have other workloads

51. Kubernetes with DC/OS 1.11 Features
Simple HA
Cluster
Provisioning
Robust API
Server Auth
Non-disruptive
Upgrades
Auto Repair
Scalable
Kubernetes
Transport Layer
Security
Cloud Native
Services

53. © 2018 Mesosphere, Inc. All Rights Reserved. 53
THANK YOU!
ANY
QUESTION
S?
@dcos
users@dcos.io
/groups/8295652
/dcos
/dcos/examples
/dcos/demos
chat.dcos.io
https://github.com/mesosphere/dcos-kubernetes-quickstart
https://mesosphere.com/blog/another-kubernetes-service/